-- Logs begin at Thu 2022-05-26 20:09:33 UTC, end at Thu 2022-05-26 21:53:37 UTC. -- May 26 21:06:02.811441 dv-cinder-828805-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 21:06:02.820360 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 21:06:02 2022] *** May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: nodename: dv-cinder-828805-3-iscsi May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: machine: x86_64 May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: clock source: unix May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: pcre jit disabled May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: detected number of CPU cores: 4 May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: current working directory: / May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: detected binary path: /usr/bin/uwsgi-core May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: your processes number limit is 47680 May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: your memory page size is 4096 bytes May 26 21:06:02.822622 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: detected max file descriptor number: 2048 May 26 21:06:02.823298 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: lock engine: pthread robust mutexes May 26 21:06:02.823298 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: thunder lock: enabled May 26 21:06:02.823298 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 21:06:02.823298 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 21:06:02.869352 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: Python main interpreter initialized at 0x55d79fd809c0 May 26 21:06:02.869352 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: python threads support enabled May 26 21:06:02.869352 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: your server socket listen backlog is limited to 100 connections May 26 21:06:02.869352 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: your mercy for graceful operations on workers is 90 seconds May 26 21:06:02.869704 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: mapped 403077 bytes (393 KB) for 2 cores May 26 21:06:02.869804 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: *** Operational MODE: preforking *** May 26 21:06:02.870087 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: *** uWSGI is running in multiple interpreter mode *** May 26 21:06:02.870087 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: spawned uWSGI master process (pid: 80744) May 26 21:06:02.870285 dv-cinder-828805-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 21:06:02.871675 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: spawned uWSGI worker 1 (pid: 80756, cores: 1) May 26 21:06:02.871675 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: spawned uWSGI worker 2 (pid: 80757, cores: 1) May 26 21:06:02.871675 dv-cinder-828805-3-iscsi devstack@keystone.service[80744]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 21:06:08.615923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 21:06:08.615923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: from cryptography.utils import int_from_bytes May 26 21:06:08.621024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 21:06:08.621024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: from cryptography.utils import int_from_bytes May 26 21:06:08.621166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 21:06:08.621166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: from cryptography.utils import int_from_bytes May 26 21:06:08.621166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 21:06:08.621166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: from cryptography.utils import int_from_bytes May 26 21:06:08.667536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 21:06:08.667727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 21:06:08.667847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 21:06:08.667944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] command line args: [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 21:06:08.668026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 21:06:08.668159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] command line args: [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 21:06:08.668235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 21:06:08.668340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ================================================================================ {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 21:06:08.668426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 21:06:08.668543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ================================================================================ {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 21:06:08.668627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] admin_token = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.668796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] admin_token = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.668876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] config_dir = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] config_dir = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] config_source = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] debug = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] config_source = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] debug = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.669897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] list_limit = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.670983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] list_limit = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] log_config_append = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] log_config_append = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] log_dir = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] log_dir = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] log_file = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] log_file = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.671964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.672974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.673927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.674835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] publish_errors = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] publish_errors = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.675983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] standard_threads = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.676945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] standard_threads = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] use_journal = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] use_journal = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] use_json = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] use_json = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] use_stderr = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.677994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] use_stderr = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.678062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] use_syslog = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.678126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] use_syslog = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.678292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.678373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 21:06:08.678544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.678637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.678777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.678859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.679888 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.680953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.681966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.682965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683346 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.683959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.684120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.684208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.684366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.684957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.685882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.686970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.687930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.688938 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.689873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.690997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.691961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.692864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.693861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.696991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.697995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.698934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.699929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.700002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.700090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.700744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.700826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.700897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.700967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.701043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.701641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.701744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.701814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.701875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.701934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.701993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702176 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.702937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.703005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.703170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.703245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.703422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.703503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.703677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.703815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.705511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] auth.external = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] auth.external = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] auth.password = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] auth.password = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.706948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] auth.token = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] auth.token = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.707860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] credential.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] credential.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.708962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.709952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710752 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.710981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.711973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.712993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.713930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.714881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.715873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716709 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.716947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717278 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.717930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.718972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.719903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.720963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721432 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.721965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.722892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.723843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.724924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.725911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.726856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.727927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.728852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.729950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.730959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.731108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.731185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.731880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.731971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.732904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.733993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.734965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.735980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.736962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] role.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] role.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737818 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.737896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] role.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] role.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.738855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.739886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.740980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.741059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.741226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.741394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.741473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.741629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.741864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.741943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.742890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.743843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.744948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.745966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.746912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.747620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.747713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.747891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.747971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.748927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.749850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.750798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.connection = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.connection = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.751973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.752995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.753922 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.754998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.755077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.755149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80757) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 21:06:08.755229 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.755456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 21:06:08.755625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80756) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 21:06:08.785897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.786118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.786204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.786344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.786431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.786583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.795574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.795799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.824570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.824785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.825014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.825302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.878557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.878702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.930405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.930619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.930786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:08.931024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.004763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.004937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.005983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.030957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.031054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.031172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.031257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.031388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.031489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.031593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.042788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.042897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.055481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.055704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.055950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.056170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.056384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.056597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.056807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80757) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.057226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.057459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.057687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.057910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.058130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.058345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.058559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80756) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 21:06:09.075612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.075856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 21:06:09.075947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.076101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 21:06:09.076222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.076374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 21:06:09.076458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.076614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 21:06:09.076694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.076877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.076959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.077126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.077204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.077372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.077458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.077620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.088499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.088585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.088775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.088856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.089038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.089120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.089313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.089394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.097982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.098092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.101285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.101376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.101557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.101638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.101831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.101911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.102107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.102223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.102373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.102461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.102622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.102740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.102995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.103073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.103264 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.103349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.103527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.103604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.103815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.103897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.104081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.104161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.131388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.131489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.133065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.133231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.133341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.133509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.138676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.138828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.138948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.139105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.139223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.139380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.146620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.146714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.146897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.146980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.151362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.151458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.151768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.151864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.152952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.153138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.153772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.153929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.154011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.173142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.173237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.173427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.173519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.178074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.178164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.181713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.181812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.181998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.182093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.187170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.187260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.190795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.190901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.194578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.194688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.194868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.194950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.195151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.195233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.195425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.195505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.195702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.195815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.195998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.196080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.215169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.215452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.216277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.216382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.216603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.216683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.216868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.217126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.224636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.225143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.226338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.226620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.226837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.227115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.229680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.229956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.230225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.230352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.230488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.230686 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.230862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.230987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.231123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.231361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.244803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.245079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.245215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.245340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.245465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.245621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.245748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.245897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.246022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.246157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.246289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.246442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.246568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.246767 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.264506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.264805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.267839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.268055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.271134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.271392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.272763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.273003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.274400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.274622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.274810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.274938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.275062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.275186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.282851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.283123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.284473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.284741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.287998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.288283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.288418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.288552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.288675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.288798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.288920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.289098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.298429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.298712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.298839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.299058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.299189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.299435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.307061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80756) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.307284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80757) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 21:06:09.307413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.307539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.307664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.307850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.307988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.308113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.308236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.308359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.308482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.308620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.308745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.308897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309146 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.309997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.310165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.310288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.310413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.310535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.310658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80756) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.310809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80757) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 21:06:09.350053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d79fd809c0 pid: 80756 (default app) May 26 21:06:09.350244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d79fd809c0 pid: 80757 (default app) May 26 21:06:09.353295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79163741-dbdc-4b3a-8298-3fbc70a04e70 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:09.353586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79163741-dbdc-4b3a-8298-3fbc70a04e70 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:09.353862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79163741-dbdc-4b3a-8298-3fbc70a04e70 None None] PATH_INFO: `/v3` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:09.358714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Thu May 26 21:06:09 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:18.346020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e3aa-f325-4a67-86d2-f86023db1ef7 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.346577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e3aa-f325-4a67-86d2-f86023db1ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.346577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e3aa-f325-4a67-86d2-f86023db1ef7 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.350959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:06:18.356164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:18.356575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:18.356710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:18.425838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: INFO dbcounter [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] Registered counter for database keystone May 26 21:06:18.446827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG oslo_db.sqlalchemy.engines [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80756) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 21:06:18.453255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writer thread running {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 21:06:18.629964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG passlib.handlers.bcrypt [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80756) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 21:06:18.630308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG passlib.handlers.bcrypt [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80756) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 21:06:18.686192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:18.687843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:18.883956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 21:06:19.035865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e3aa-f325-4a67-86d2-f86023db1ef7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.106736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: INFO dbcounter [None req-e792e3aa-f325-4a67-86d2-f86023db1ef7 None None] Registered counter for database keystone May 26 21:06:19.126083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG oslo_db.sqlalchemy.engines [None req-e792e3aa-f325-4a67-86d2-f86023db1ef7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80757) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 21:06:19.132266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writer thread running {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 21:06:19.254643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e3aa-f325-4a67-86d2-f86023db1ef7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.258893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63439f-7f3e-4e14-b95b-90439a03f6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.259620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63439f-7f3e-4e14-b95b-90439a03f6df admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.259915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63439f-7f3e-4e14-b95b-90439a03f6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.260192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63439f-7f3e-4e14-b95b-90439a03f6df admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.268837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63439f-7f3e-4e14-b95b-90439a03f6df admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.320692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8f63439f-7f3e-4e14-b95b-90439a03f6df admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 21:06:19.321033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63439f-7f3e-4e14-b95b-90439a03f6df admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.331797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Thu May 26 21:06:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:06:19.337879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:19.353044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1378c60-98bb-4a3d-b4a2-fc918e51a7c2 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:19.357501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a4a48-b8bf-4dc4-a516-9ab7ee206722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:19.358625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a4a48-b8bf-4dc4-a516-9ab7ee206722 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:19.358913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a4a48-b8bf-4dc4-a516-9ab7ee206722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:19.359189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a4a48-b8bf-4dc4-a516-9ab7ee206722 admin admin] PATH_INFO: `/v3/projects/213a059b002f40c1bddeb6d9f1da1459` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:19.360723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a4a48-b8bf-4dc4-a516-9ab7ee206722 admin admin] RBAC: Authorizing `identity:get_project(project_id=213a059b002f40c1bddeb6d9f1da1459)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:19.520004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b56a4a48-b8bf-4dc4-a516-9ab7ee206722 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 21:06:19.520248 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a4a48-b8bf-4dc4-a516-9ab7ee206722 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:19.524909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Thu May 26 21:06:19 2022] GET /identity/v3/projects/213a059b002f40c1bddeb6d9f1da1459 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:21.701537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9954d-814c-42dd-ad65-ef014a4facc0 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.702052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9954d-814c-42dd-ad65-ef014a4facc0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.702136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b9954d-814c-42dd-ad65-ef014a4facc0 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.706265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:21.711665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954fec-9d70-49d6-82ac-3735972fa14a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.712097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954fec-9d70-49d6-82ac-3735972fa14a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.712254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f954fec-9d70-49d6-82ac-3735972fa14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.731508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6f954fec-9d70-49d6-82ac-3735972fa14a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:21.733060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6f954fec-9d70-49d6-82ac-3735972fa14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:21.740956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 21:06:21.763377 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9954d-814c-42dd-ad65-ef014a4facc0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.775334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b9954d-814c-42dd-ad65-ef014a4facc0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.780153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f288e9-5567-4bb0-b011-97e2e3aec176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.780781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f288e9-5567-4bb0-b011-97e2e3aec176 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.781069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f288e9-5567-4bb0-b011-97e2e3aec176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.781347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f288e9-5567-4bb0-b011-97e2e3aec176 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.788945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f288e9-5567-4bb0-b011-97e2e3aec176 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.790707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f288e9-5567-4bb0-b011-97e2e3aec176 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.800026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Thu May 26 21:06:21 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:06:21.805337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954fec-9d70-49d6-82ac-3735972fa14a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:21.814940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f954fec-9d70-49d6-82ac-3735972fa14a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:21.819403 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2009a8-3b37-4ebf-85e2-d7b0be1c2a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:21.820340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2009a8-3b37-4ebf-85e2-d7b0be1c2a9c admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:21.820593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2009a8-3b37-4ebf-85e2-d7b0be1c2a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:21.820873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2009a8-3b37-4ebf-85e2-d7b0be1c2a9c admin admin] PATH_INFO: `/v3/users/ef098ca75a4b4231b612fc38df12339d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:21.822521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2009a8-3b37-4ebf-85e2-d7b0be1c2a9c admin admin] RBAC: Authorizing `identity:get_user(user_id=ef098ca75a4b4231b612fc38df12339d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:21.824418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2009a8-3b37-4ebf-85e2-d7b0be1c2a9c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:21.825975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Thu May 26 21:06:21 2022] GET /identity/v3/users/ef098ca75a4b4231b612fc38df12339d => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:24.078081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531509b-76f8-4903-b512-cb03c5225676 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.078580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531509b-76f8-4903-b512-cb03c5225676 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.078580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531509b-76f8-4903-b512-cb03c5225676 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.079956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:06:24.085335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54265cc-ded9-4594-8dfb-4ca014487d8e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.085745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54265cc-ded9-4594-8dfb-4ca014487d8e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.085901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54265cc-ded9-4594-8dfb-4ca014487d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.093315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17baee-b9cd-4d87-9537-8f57ffdbc771 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.093604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17baee-b9cd-4d87-9537-8f57ffdbc771 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.093882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17baee-b9cd-4d87-9537-8f57ffdbc771 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.095194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:24.100398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.100686 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.100964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.102676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e54265cc-ded9-4594-8dfb-4ca014487d8e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:24.103897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e54265cc-ded9-4594-8dfb-4ca014487d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:24.141735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG passlib.handlers.bcrypt [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80757) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 21:06:24.142168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG passlib.handlers.bcrypt [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80757) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 21:06:24.184082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:24.189689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:24.191396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:24.207135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54265cc-ded9-4594-8dfb-4ca014487d8e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.230814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:24.294691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:24.312809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54265cc-ded9-4594-8dfb-4ca014487d8e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.320022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.350317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.351135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.351489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.351867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.362825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:24.374906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:24.376201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:24.377047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:06:24.382522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.470263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:24.488672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39ce27-1fe0-4fc3-82a1-7b654b52c60b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.523572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.524268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.524553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.524835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.525575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:24.525884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:24.527291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:24.536562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a4df9-3ae2-4e49-a599-bb181db58066 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.538894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:24.539312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:24.545299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.578186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.579095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.579396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.579797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.647021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48316e14-c4dd-4f5c-a00b-3cf0eedf202a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.651251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:24.653458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:24.681854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.682760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.683039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.683312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] PATH_INFO: `/v3/users/ef098ca75a4b4231b612fc38df12339d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.685136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] RBAC: Authorizing `identity:get_user(user_id=ef098ca75a4b4231b612fc38df12339d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:24.686588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:24.688314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 9/16] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/users/ef098ca75a4b4231b612fc38df12339d => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:24.693562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.710329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:24.758428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 8/17] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 21:06:24.763942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.766604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:24.794879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93712b8-90ff-4cf9-8b0b-379e9a37f79d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.829292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.830086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.830368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.830647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.831841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:24.833295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:24.835043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:24.840653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.864644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a124dd6-1471-4794-ae29-6c88d0db1b18 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.897614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.898340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.898624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.898899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.917985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:24.920058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:24.920623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:24.939419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9ed6c-3185-473b-af0b-bb8a120a7c02 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:06:24.940471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:24.941257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:24.942045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:06:24.942045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:06:24.942045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:06:24.942045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:24.942045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:06:24.948773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:24.974893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486642a-321a-4c5a-ab11-1626b737d581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:24.975558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486642a-321a-4c5a-ab11-1626b737d581 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:24.975889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486642a-321a-4c5a-ab11-1626b737d581 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:24.976174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486642a-321a-4c5a-ab11-1626b737d581 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:24.977015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486642a-321a-4c5a-ab11-1626b737d581 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.domain.id=default, user.id=ef098ca75a4b4231b612fc38df12339d) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:24.977327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486642a-321a-4c5a-ab11-1626b737d581 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:24.978790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486642a-321a-4c5a-ab11-1626b737d581 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:24.986049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/role_assignments?user.id=ef098ca75a4b4231b612fc38df12339d&scope.domain.id=default&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:25.009926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:25.060208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51158132-b176-41fb-9bdf-d261f8e7edb4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:25.096817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee35d2-5259-4d32-8866-51ea00f61e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:25.097507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee35d2-5259-4d32-8866-51ea00f61e67 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:25.097805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee35d2-5259-4d32-8866-51ea00f61e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:25.098083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee35d2-5259-4d32-8866-51ea00f61e67 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:25.098826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee35d2-5259-4d32-8866-51ea00f61e67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:25.099139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee35d2-5259-4d32-8866-51ea00f61e67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:25.100593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee35d2-5259-4d32-8866-51ea00f61e67 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:25.120228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:27.373727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff6633-b34d-43b2-a1c8-7e314ec27684 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:27.374276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff6633-b34d-43b2-a1c8-7e314ec27684 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:27.374276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff6633-b34d-43b2-a1c8-7e314ec27684 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:27.375696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:27.381054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6bc1d-6be7-4a14-965c-f476a4cae9c1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:27.381449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6bc1d-6be7-4a14-965c-f476a4cae9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:27.381662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6bc1d-6be7-4a14-965c-f476a4cae9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:27.404711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b0e6bc1d-6be7-4a14-965c-f476a4cae9c1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:27.405918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b0e6bc1d-6be7-4a14-965c-f476a4cae9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:27.493885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 11/23] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:27.534769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff6633-b34d-43b2-a1c8-7e314ec27684 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:27.693121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6597-9bdb-45eb-aa66-18e8731a08a2 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:27.699057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6597-9bdb-45eb-aa66-18e8731a08a2 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:27.699639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d6597-9bdb-45eb-aa66-18e8731a08a2 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:27.709047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 12/24] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:27 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:27.713204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff6633-b34d-43b2-a1c8-7e314ec27684 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:27.720202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53a228-b466-45d7-962a-dd1741e353c0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:27.720753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53a228-b466-45d7-962a-dd1741e353c0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:27.721230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c53a228-b466-45d7-962a-dd1741e353c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:27.746395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3c53a228-b466-45d7-962a-dd1741e353c0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:27.748025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3c53a228-b466-45d7-962a-dd1741e353c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:27.817951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:27.818779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:27.819068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:27.819347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:27.820173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:27.846950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:27.852251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 13/25] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:27.853876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b2542-b0ed-40d9-9e1d-dd2f53cbdd6f None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:27.854361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b2542-b0ed-40d9-9e1d-dd2f53cbdd6f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:27.854831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b2542-b0ed-40d9-9e1d-dd2f53cbdd6f None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:27.855310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:27.856176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:27.857006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 13/26] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:27.859202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 14/27] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:27.863120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:27.864623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16df8-9f13-4283-92e0-9010a23983ac None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:27.865262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16df8-9f13-4283-92e0-9010a23983ac None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:27.865734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a16df8-9f13-4283-92e0-9010a23983ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:27.872345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:27.909517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-01a16df8-9f13-4283-92e0-9010a23983ac None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:27.910779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-01a16df8-9f13-4283-92e0-9010a23983ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:28.007809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3ce72-34e4-4aab-94c0-9f39d1dda398 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:28.050241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:28.050923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.051212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.051492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.052275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:28.052586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:28.054007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:28.068719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 15/28] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:28.069910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16df8-9f13-4283-92e0-9010a23983ac None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:28.071495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 14/29] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:28.077639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:28.205617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a16df8-9f13-4283-92e0-9010a23983ac None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:28.214004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:28.228121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:28.247312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:28.251300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.251612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.251925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.277878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:28.283080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:28.287789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c9e17-9a08-46c4-9527-721acfab6a0e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:06:28.319718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:28.320598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:28.321411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:06:28.321411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:06:28.321411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:06:28.321411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:28.325780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 16/30] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:28.326604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:28.354459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:28.355359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.355647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.356493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] PATH_INFO: `/v3/users/ef098ca75a4b4231b612fc38df12339d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.364738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] RBAC: Authorizing `identity:get_user(user_id=ef098ca75a4b4231b612fc38df12339d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:28.366817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:28.369042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 15/31] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:28 2022] GET /identity/v3/users/ef098ca75a4b4231b612fc38df12339d => generated 247 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:28.370174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:28.467486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e56816-0113-4783-9e42-9400ef30e94f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:28.510511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769bb20-759d-4cb5-bae5-85c2750525b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:28.512549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769bb20-759d-4cb5-bae5-85c2750525b6 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.512838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769bb20-759d-4cb5-bae5-85c2750525b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.513117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769bb20-759d-4cb5-bae5-85c2750525b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.513881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769bb20-759d-4cb5-bae5-85c2750525b6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:28.519124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3769bb20-759d-4cb5-bae5-85c2750525b6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:28.563989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a697a5-ee63-4db7-b03a-d040d79e5c3b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:28.601717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:28.602400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.602689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.602977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.603718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:28.604094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:28.605526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:28.607493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:28.625524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 16/32] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:28.630381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:28.744646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d17723-212d-4635-a426-45d23e9129ea None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:28.776145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 17/33] 10.222.0.42 () {60 vars in 1131 bytes} [Thu May 26 21:06:28 2022] POST /identity/v3/roles => generated 219 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:28.777923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a115-6b1e-403c-ab3b-d5c711c57006 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.778232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a115-6b1e-403c-ab3b-d5c711c57006 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.778509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88a115-6b1e-403c-ab3b-d5c711c57006 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.779861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 18/34] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:28.780586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88a115-6b1e-403c-ab3b-d5c711c57006 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:28.782251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3b4981-7f4b-49b7-b31a-769b2553bc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:28.783068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b4981-7f4b-49b7-b31a-769b2553bc08 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.783358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b4981-7f4b-49b7-b31a-769b2553bc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.783778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3b4981-7f4b-49b7-b31a-769b2553bc08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.785128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b4981-7f4b-49b7-b31a-769b2553bc08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:28.786633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3b4981-7f4b-49b7-b31a-769b2553bc08 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:28.788889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 17/35] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:28.790250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f50131-e194-4c5d-a328-da56b3f0fd04 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.790665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f50131-e194-4c5d-a328-da56b3f0fd04 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.790953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f50131-e194-4c5d-a328-da56b3f0fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.802797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:28.812127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c7f50131-e194-4c5d-a328-da56b3f0fd04 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:28.813661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c7f50131-e194-4c5d-a328-da56b3f0fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:28.882710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:28.896583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 18/36] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:28.897972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f50131-e194-4c5d-a328-da56b3f0fd04 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:28.899008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88a115-6b1e-403c-ab3b-d5c711c57006 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:28.961437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:28.962120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:28.962483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:28.962773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:28.963796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:28.965960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:29.036729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f50131-e194-4c5d-a328-da56b3f0fd04 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:29.082953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd873f1-eabe-4fd3-b4b2-e2d93aa0f0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:29.083904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd873f1-eabe-4fd3-b4b2-e2d93aa0f0fc None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.084189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd873f1-eabe-4fd3-b4b2-e2d93aa0f0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.084465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd873f1-eabe-4fd3-b4b2-e2d93aa0f0fc None admin] PATH_INFO: `/v3/domains/default/users/ef098ca75a4b4231b612fc38df12339d/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.086590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd873f1-eabe-4fd3-b4b2-e2d93aa0f0fc None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ef098ca75a4b4231b612fc38df12339d, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:29.088297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd873f1-eabe-4fd3-b4b2-e2d93aa0f0fc None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:29.160269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 19/37] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:06:28 2022] POST /identity/v3/projects => generated 301 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:29.161019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:29.257435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:29.296905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88470e6e-0ac6-4a93-9537-7b31d9df21d8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:29.337509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2268e4-5b6c-451a-86e1-dfb471b3568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:29.338201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2268e4-5b6c-451a-86e1-dfb471b3568e None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.338533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2268e4-5b6c-451a-86e1-dfb471b3568e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.338825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2268e4-5b6c-451a-86e1-dfb471b3568e None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.339616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2268e4-5b6c-451a-86e1-dfb471b3568e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:29.341713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2268e4-5b6c-451a-86e1-dfb471b3568e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:29.360342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 19/38] 10.222.0.42 () {58 vars in 1361 bytes} [Thu May 26 21:06:28 2022] PUT /identity/v3/domains/default/users/ef098ca75a4b4231b612fc38df12339d/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:29.362244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79e1cf-c7b7-48e4-a7ce-5ae1f1c53dba None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.362540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79e1cf-c7b7-48e4-a7ce-5ae1f1c53dba None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.362822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c79e1cf-c7b7-48e4-a7ce-5ae1f1c53dba None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.367286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 20/39] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:29.374260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95175e-1f71-4da0-8bfc-f46ae611e00b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.374877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95175e-1f71-4da0-8bfc-f46ae611e00b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.375322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95175e-1f71-4da0-8bfc-f46ae611e00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.398647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1b95175e-1f71-4da0-8bfc-f46ae611e00b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:29.401059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1b95175e-1f71-4da0-8bfc-f46ae611e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:29.405172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 20/40] 10.222.0.42 () {60 vars in 1131 bytes} [Thu May 26 21:06:29 2022] POST /identity/v3/roles => generated 225 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:29.467697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bae94-849a-464f-9f0f-237847a30c70 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.468345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bae94-849a-464f-9f0f-237847a30c70 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.468345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0bae94-849a-464f-9f0f-237847a30c70 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.468653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:29.470116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 21/41] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:29.478009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178cc8e-491d-4f3e-b5d5-a7bf66ce8a1a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.478316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178cc8e-491d-4f3e-b5d5-a7bf66ce8a1a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.478606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178cc8e-491d-4f3e-b5d5-a7bf66ce8a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.494961 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 21/42] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:29.501845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7178cc8e-491d-4f3e-b5d5-a7bf66ce8a1a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:29.503388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7178cc8e-491d-4f3e-b5d5-a7bf66ce8a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:29.522726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95175e-1f71-4da0-8bfc-f46ae611e00b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:29.585708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 22/43] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:29.609228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178cc8e-491d-4f3e-b5d5-a7bf66ce8a1a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:29.657118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95175e-1f71-4da0-8bfc-f46ae611e00b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:29.681908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:29.704360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40144c39-c99b-4b5b-a7d3-027f8939b099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:29.705660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40144c39-c99b-4b5b-a7d3-027f8939b099 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.705660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40144c39-c99b-4b5b-a7d3-027f8939b099 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.705883 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40144c39-c99b-4b5b-a7d3-027f8939b099 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.706546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40144c39-c99b-4b5b-a7d3-027f8939b099 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:29.709213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40144c39-c99b-4b5b-a7d3-027f8939b099 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:29.740396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178cc8e-491d-4f3e-b5d5-a7bf66ce8a1a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:29.774633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:29.775429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.775713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.776017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.777174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:29.778709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:29.780382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 23/44] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:29.785769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:29.816837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 22/45] 10.222.0.42 () {60 vars in 1131 bytes} [Thu May 26 21:06:29 2022] POST /identity/v3/roles => generated 223 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:29.897456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5cc4d-d97b-4a53-a2d7-a668d3de8f43 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:29.929788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:29.939130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f3e86-c417-4a4e-821b-7c352231310b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:29.939840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f3e86-c417-4a4e-821b-7c352231310b None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:29.940131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f3e86-c417-4a4e-821b-7c352231310b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:29.940412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f3e86-c417-4a4e-821b-7c352231310b None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:29.941143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f3e86-c417-4a4e-821b-7c352231310b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:29.943248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f3e86-c417-4a4e-821b-7c352231310b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:30.051257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:06:29 2022] POST /identity/v3/projects => generated 312 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:31.691012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf5011-af42-4803-b89a-bc69ad7919f8 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:31.691537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf5011-af42-4803-b89a-bc69ad7919f8 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:31.691537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf5011-af42-4803-b89a-bc69ad7919f8 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:31.693238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 23/47] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:31.699110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac542-2560-4328-afa7-e6ff96a4ac56 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:31.699557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac542-2560-4328-afa7-e6ff96a4ac56 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:31.700269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbac542-2560-4328-afa7-e6ff96a4ac56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:31.717637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bcbac542-2560-4328-afa7-e6ff96a4ac56 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:31.718806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bcbac542-2560-4328-afa7-e6ff96a4ac56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:31.786637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 25/48] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:31.808875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf5011-af42-4803-b89a-bc69ad7919f8 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:31.855641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:31.913342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf5011-af42-4803-b89a-bc69ad7919f8 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:31.947213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:31.947972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:31.948259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:31.948536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:31.959086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:31.967347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:31.968322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:31.969135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 24/49] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:31.974496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac542-2560-4328-afa7-e6ff96a4ac56 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:32.077133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbac542-2560-4328-afa7-e6ff96a4ac56 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:32.088374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:32.111538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e609c-ebda-4e7a-a0d6-516d1b15aa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:32.112239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e609c-ebda-4e7a-a0d6-516d1b15aa74 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.112549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e609c-ebda-4e7a-a0d6-516d1b15aa74 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.112807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e609c-ebda-4e7a-a0d6-516d1b15aa74 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.113588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e609c-ebda-4e7a-a0d6-516d1b15aa74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:32.113866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e609c-ebda-4e7a-a0d6-516d1b15aa74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:32.115249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e609c-ebda-4e7a-a0d6-516d1b15aa74 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:32.127776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 26/50] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:32.133435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:32.233347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2490a913-c88a-4b5f-a5d3-37fa106ed73a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:32.234799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:32.246776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9bba-03f0-44b0-9c8c-429352c0f9ff None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.247178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9bba-03f0-44b0-9c8c-429352c0f9ff None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.248585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d9bba-03f0-44b0-9c8c-429352c0f9ff None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.248921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 27/51] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:32.253944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f32bf-20db-47fe-a8da-16ca8f101098 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.254304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f32bf-20db-47fe-a8da-16ca8f101098 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.254585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727f32bf-20db-47fe-a8da-16ca8f101098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.267875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:32.268757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.269049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.269336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] PATH_INFO: `/v3/users/ef098ca75a4b4231b612fc38df12339d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.270590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] RBAC: Authorizing `identity:get_user(user_id=ef098ca75a4b4231b612fc38df12339d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:32.271276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-727f32bf-20db-47fe-a8da-16ca8f101098 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:32.272099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:32.272495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-727f32bf-20db-47fe-a8da-16ca8f101098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:32.273740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 25/52] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:32 2022] GET /identity/v3/users/ef098ca75a4b4231b612fc38df12339d => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:32.279789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:32.342283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 28/53] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:32.365358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f32bf-20db-47fe-a8da-16ca8f101098 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:32.384013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed2abc-c2a0-4699-8b80-53b7418ed4c0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:32.419916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:32.420707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.420992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.421295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.422607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:32.424112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:32.426029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 26/54] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:32.431539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:32.468140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727f32bf-20db-47fe-a8da-16ca8f101098 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:32.469716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:32.471373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:32.505776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:32.506506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.506793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.507076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.508299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:32.509752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:32.511424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 29/55] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:32.517607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:32.535597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0ab82-7586-4db1-a632-4971bab05c82 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:32.568911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dd2ba2-38dc-46a3-bf66-d1818e0c820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:32.569564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2ba2-38dc-46a3-bf66-d1818e0c820f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.569848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2ba2-38dc-46a3-bf66-d1818e0c820f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.570123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dd2ba2-38dc-46a3-bf66-d1818e0c820f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.570955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2ba2-38dc-46a3-bf66-d1818e0c820f None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.domain.id=default, user.id=ef098ca75a4b4231b612fc38df12339d) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:32.571265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2ba2-38dc-46a3-bf66-d1818e0c820f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:32.573105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dd2ba2-38dc-46a3-bf66-d1818e0c820f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:32.579213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 27/56] 10.222.0.42 () {56 vars in 1322 bytes} [Thu May 26 21:06:32 2022] GET /identity/v3/role_assignments?user.id=ef098ca75a4b4231b612fc38df12339d&scope.domain.id=default&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:32.631478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd18c1-9c29-495b-bf5c-60c17ef3b1b9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:32.669750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ec1e7-b619-4630-9beb-b9d09fad9eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:32.670428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ec1e7-b619-4630-9beb-b9d09fad9eda None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:32.671511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ec1e7-b619-4630-9beb-b9d09fad9eda None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:32.671511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ec1e7-b619-4630-9beb-b9d09fad9eda None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:32.671689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ec1e7-b619-4630-9beb-b9d09fad9eda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:32.673179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ec1e7-b619-4630-9beb-b9d09fad9eda None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:32.816703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 30/57] 10.222.0.42 () {60 vars in 1140 bytes} [Thu May 26 21:06:32 2022] POST /identity/v3/projects => generated 298 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:35.016537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b9a2-b563-4f0e-9f25-b2b721ff7e10 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:35.017028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b9a2-b563-4f0e-9f25-b2b721ff7e10 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:35.017028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0b9a2-b563-4f0e-9f25-b2b721ff7e10 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:35.018400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 28/58] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:35.023669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb438-f93d-41bb-9bea-ac518997f67b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:35.024113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb438-f93d-41bb-9bea-ac518997f67b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:35.024265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765bb438-f93d-41bb-9bea-ac518997f67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:35.040546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-765bb438-f93d-41bb-9bea-ac518997f67b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:35.041726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-765bb438-f93d-41bb-9bea-ac518997f67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:35.058299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:35.107338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 31/59] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:35.132551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0b9a2-b563-4f0e-9f25-b2b721ff7e10 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:35.229724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0b9a2-b563-4f0e-9f25-b2b721ff7e10 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:35.231753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:35.263382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a7b628-9409-44b5-aed7-7af487cddbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:35.264197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7b628-9409-44b5-aed7-7af487cddbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:35.264497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7b628-9409-44b5-aed7-7af487cddbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:35.264969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a7b628-9409-44b5-aed7-7af487cddbf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:35.266655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7b628-9409-44b5-aed7-7af487cddbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:35.268179 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a7b628-9409-44b5-aed7-7af487cddbf0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:35.269810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 29/60] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:35.275241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bb438-f93d-41bb-9bea-ac518997f67b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:35.371083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765bb438-f93d-41bb-9bea-ac518997f67b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:35.404522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e6f840-a24a-419b-9c43-6497fb86b12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:35.405190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6f840-a24a-419b-9c43-6497fb86b12f None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:35.405473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6f840-a24a-419b-9c43-6497fb86b12f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:35.405748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e6f840-a24a-419b-9c43-6497fb86b12f None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:35.406460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6f840-a24a-419b-9c43-6497fb86b12f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:35.407929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e6f840-a24a-419b-9c43-6497fb86b12f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:35.516891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 32/61] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:06:35 2022] POST /identity/v3/users => generated 275 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:38.802612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1348f62-2f01-457e-9db3-ab9c0a0982cb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:38.803550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1348f62-2f01-457e-9db3-ab9c0a0982cb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:38.804021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1348f62-2f01-457e-9db3-ab9c0a0982cb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:38.805675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 30/62] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:38.811909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577cb0-851f-4e1c-9c84-140c08a978ce None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:38.812798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577cb0-851f-4e1c-9c84-140c08a978ce None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:38.813231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a577cb0-851f-4e1c-9c84-140c08a978ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:38.836835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3a577cb0-851f-4e1c-9c84-140c08a978ce None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:38.838320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3a577cb0-851f-4e1c-9c84-140c08a978ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:38.841327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:38.933244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 33/63] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:38.981913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1348f62-2f01-457e-9db3-ab9c0a0982cb None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.024063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43cd7f-d3e4-4651-959f-23805f18933b None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.024395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43cd7f-d3e4-4651-959f-23805f18933b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.024624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43cd7f-d3e4-4651-959f-23805f18933b None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.026002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 34/64] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:06:39.031392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877887d-f265-47e9-bfaf-407d0a154332 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877887d-f265-47e9-bfaf-407d0a154332 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.031994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877887d-f265-47e9-bfaf-407d0a154332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.060020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b877887d-f265-47e9-bfaf-407d0a154332 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:39.061550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b877887d-f265-47e9-bfaf-407d0a154332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:39.140614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1348f62-2f01-457e-9db3-ab9c0a0982cb None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.153041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 35/65] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:39.154677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c5f4-de5f-4820-922b-8eacc46a8a0c None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.155186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c5f4-de5f-4820-922b-8eacc46a8a0c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.155784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b6c5f4-de5f-4820-922b-8eacc46a8a0c None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.157261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 36/66] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:39.165132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57eb944-dd1c-4f79-ae06-654535b5feb0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.165806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57eb944-dd1c-4f79-ae06-654535b5feb0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.166409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57eb944-dd1c-4f79-ae06-654535b5feb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.184708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d57eb944-dd1c-4f79-ae06-654535b5feb0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:39.186239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d57eb944-dd1c-4f79-ae06-654535b5feb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:39.207895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.208706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.208996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.209279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.223665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:39.233918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:39.234840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:39.235631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:39.238598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 31/67] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:39.243463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.266855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:39.272411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 37/68] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:39.280058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57eb944-dd1c-4f79-ae06-654535b5feb0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.285846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:39.383807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a373736-21e5-432e-bcd3-68b63445adce None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.407006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57eb944-dd1c-4f79-ae06-654535b5feb0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.429242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.430805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.431110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.431396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.447985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.448306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:39.449294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.449822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.450296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.451284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:39.451856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:39.453529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:39.457194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:39.458094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:39.458829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 32/69] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:39.465680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.469777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 38/70] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:39.470827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.537586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:39.571503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:39.582843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1044c7de-a313-4de4-a40f-0182e83e0a1c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.586240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb0d217-9018-4d44-8e18-a3b0b72572a6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.621016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1931891-0278-48cb-a561-156cd989ceff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.621995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1931891-0278-48cb-a561-156cd989ceff None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.622288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1931891-0278-48cb-a561-156cd989ceff None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.622636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1931891-0278-48cb-a561-156cd989ceff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.624052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.624819 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.625112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.625397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.626169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:39.626482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:39.627896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:39.634329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1931891-0278-48cb-a561-156cd989ceff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:39.636789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 39/71] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:39.638258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f1931891-0278-48cb-a561-156cd989ceff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:39.643040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:39.643992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:39.645047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 33/72] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:39.645987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f25d-7b12-4f5f-8cc9-e338dfd6a530 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.645987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f25d-7b12-4f5f-8cc9-e338dfd6a530 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.646108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f25d-7b12-4f5f-8cc9-e338dfd6a530 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.647419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 34/73] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:39.648347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957de9aa-2fbb-4ad3-8452-95dd3f01e645 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.648670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957de9aa-2fbb-4ad3-8452-95dd3f01e645 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.648947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957de9aa-2fbb-4ad3-8452-95dd3f01e645 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.650097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 35/74] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:39.650785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957de9aa-2fbb-4ad3-8452-95dd3f01e645 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.748872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5973a0-b3fb-4b90-adb8-be354303a730 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.755604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957de9aa-2fbb-4ad3-8452-95dd3f01e645 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.770581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:39.781860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25277e9b-c6d8-4ff7-b065-ee0cac481efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.782792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25277e9b-c6d8-4ff7-b065-ee0cac481efa None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.783854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25277e9b-c6d8-4ff7-b065-ee0cac481efa None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.783854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25277e9b-c6d8-4ff7-b065-ee0cac481efa None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.788947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.789764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.790061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.790376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] PATH_INFO: `/v3/users/ef098ca75a4b4231b612fc38df12339d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.791586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] RBAC: Authorizing `identity:get_user(user_id=ef098ca75a4b4231b612fc38df12339d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:39.793605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:39.795397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 36/75] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/users/ef098ca75a4b4231b612fc38df12339d => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:39.796204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.800926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25277e9b-c6d8-4ff7-b065-ee0cac481efa None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:39.802634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25277e9b-c6d8-4ff7-b065-ee0cac481efa None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:39.808061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 40/76] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:39.810084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c18e7-41fb-42dc-964c-39c3cc0f9df1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.810372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c18e7-41fb-42dc-964c-39c3cc0f9df1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.810651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392c18e7-41fb-42dc-964c-39c3cc0f9df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.817224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:39.828346 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-392c18e7-41fb-42dc-964c-39c3cc0f9df1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:39.829543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-392c18e7-41fb-42dc-964c-39c3cc0f9df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:39.895364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 41/77] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:39.896785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b093617-0087-44b5-9177-212c113fff38 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.897082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b093617-0087-44b5-9177-212c113fff38 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.897363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b093617-0087-44b5-9177-212c113fff38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.898923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105b458-77e0-4230-a6b6-198dd0c5e61a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:39.915034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5b093617-0087-44b5-9177-212c113fff38 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:39.916250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5b093617-0087-44b5-9177-212c113fff38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:39.937082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:39.937757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:39.938043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:39.938328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:39.939084 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:39.939393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:39.940827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:39.949228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 37/78] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:39.949881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:39.987777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 42/79] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:39.988974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b093617-0087-44b5-9177-212c113fff38 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.031641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.032095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.050872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06aa92-0815-485a-9b39-2461e85507f1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.082770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.083474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.083842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.084065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.091551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b093617-0087-44b5-9177-212c113fff38 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.122300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.123356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.123648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.123956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.150311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.151815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.191549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.193047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.229380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.234440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.249318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 38/80] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:40.253961 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.299937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 43/81] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:40.301094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.354617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107cea10-12e1-4f68-b928-4739d012eb97 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.388934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.389718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.390000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.390276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.400036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:40.400174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac0bd27-1c13-43f8-94bb-57bcc9cfcbf2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:40.408445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:40.409568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:40.410469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 39/82] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:40.411614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.437909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.438794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.439111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.439358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.440596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.442120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.443710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 44/83] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:40 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:40.443710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.455908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.458164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.514774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0c164-57ed-4274-9565-191668c64b92 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.544340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20027ec2-88a8-4758-bac7-ff9f17a96a54 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.549080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.549872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.550156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.550435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.551682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.553122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.554730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 40/84] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:40.558811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.579200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.580113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.580400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.580680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.581892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.583347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.585067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 45/85] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:40 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:40.585838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.662076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ce97a-cdcc-4459-92f1-9bc581f04bae None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.664880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.671887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.687554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd4753-5920-4c2f-9d5e-0ca47f8ffbd3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.699867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.700748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.701038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.701333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.702561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.704064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.705776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 41/86] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:40 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:40.706556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.727089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.727881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.728203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.728512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.729295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:40.729623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.731652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.744602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 46/87] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:40.744602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:40.810655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4630be88-6306-4d1e-afa8-5b90b6216d5a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.839547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16a7266-a48c-4fe7-b767-45c451998064 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:40.844043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.844692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.844976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.845256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.873724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:40.874406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:40.874732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:40.875041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:40.875795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.877229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:40.885931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.889620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:40.911801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:40.913479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:41.002535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 47/88] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:06:40 2022] POST /identity/v3/users => generated 289 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:41.003473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:41.018174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 42/89] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:41.019257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:41.115401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:41.136011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a557b-def1-4e26-98d1-4042e0565948 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:41.142364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2041fe4-61c1-416d-b676-12be7856c735 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:41.174520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:41.175222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.175542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.175887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.176765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=ef098ca75a4b4231b612fc38df12339d) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:41.177092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:41.178568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:41.181239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:41.181907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.182195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.182477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.183316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=591c45f7bb064291b85f4b4a3998abe6) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:41.183632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:41.185680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:41.189080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 48/90] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:41 2022] GET /identity/v3/role_assignments?user.id=ef098ca75a4b4231b612fc38df12339d&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:41.189980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:41.192365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 43/91] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:41 2022] GET /identity/v3/role_assignments?user.id=591c45f7bb064291b85f4b4a3998abe6&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=1183a502796d46e580a92e3c20f04103 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:41.193215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:41.286956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:41.345904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164ff5b-d76e-47b6-b43e-5ef8fefb8406 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:41.352659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6cb95f-0ac8-4843-8b69-bffa915b779a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:41.397424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:41.398404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.398696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.398980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.400776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:41.403343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:41.405453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 44/92] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:41 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:41.407355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:41.408529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.409022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.409501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.411110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:41.411478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:41.413162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:41.419909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 49/93] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:41 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:41.424838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:41.425131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:41.538918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce84ce9-4dcc-4781-8bfa-117b10486cb0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:41.564114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4a44-3309-4b8e-9dc3-d77c6666c10e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:41.583071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:41.584523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.585131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.585621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.586817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] RBAC: Adding query filter params (role.id=c23de713167345639a2085add0a54ae6, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=591c45f7bb064291b85f4b4a3998abe6) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:41.587332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:41.589756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:41.599538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 45/94] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:41 2022] GET /identity/v3/role_assignments?user.id=591c45f7bb064291b85f4b4a3998abe6&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=c23de713167345639a2085add0a54ae6 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:41.603277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:41.622252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:41.623008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:41.623290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:41.623500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:41.747985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:41.749501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:41.860381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 50/95] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:41.866975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:41.976029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:42.016093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11f113-4298-4cb5-bc70-7c95d5d926e4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:42.060520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbec1f-bdb9-40f6-83e2-7d69b2bc91eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:42.061723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbec1f-bdb9-40f6-83e2-7d69b2bc91eb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:42.062260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbec1f-bdb9-40f6-83e2-7d69b2bc91eb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:42.062736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbec1f-bdb9-40f6-83e2-7d69b2bc91eb None admin] PATH_INFO: `/v3/projects/c515f5d4ee914043984885a852baa240` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:42.064139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbec1f-bdb9-40f6-83e2-7d69b2bc91eb None admin] RBAC: Authorizing `identity:get_project(project_id=c515f5d4ee914043984885a852baa240)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:42.065797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbec1f-bdb9-40f6-83e2-7d69b2bc91eb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:42.071282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 46/96] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:41 2022] GET /identity/v3/projects/c515f5d4ee914043984885a852baa240 => generated 312 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:42.074799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:42.101667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:42.197049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee25935-323d-4607-ad34-47a3dac69c0e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:42.233921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb2536-2159-475b-ba4b-a1c8c607b3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:42.235014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2536-2159-475b-ba4b-a1c8c607b3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:42.235509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2536-2159-475b-ba4b-a1c8c607b3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:42.236004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb2536-2159-475b-ba4b-a1c8c607b3f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:42.237028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2536-2159-475b-ba4b-a1c8c607b3f9 None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=c515f5d4ee914043984885a852baa240, user.id=591c45f7bb064291b85f4b4a3998abe6) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:42.237577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2536-2159-475b-ba4b-a1c8c607b3f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:42.239274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb2536-2159-475b-ba4b-a1c8c607b3f9 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:42.245849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 51/97] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:42 2022] GET /identity/v3/role_assignments?user.id=591c45f7bb064291b85f4b4a3998abe6&scope.project.id=c515f5d4ee914043984885a852baa240&role.id=1183a502796d46e580a92e3c20f04103 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:44.216072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155badb-cc68-4417-8e22-b9b956eeb26f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.216597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155badb-cc68-4417-8e22-b9b956eeb26f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.216903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155badb-cc68-4417-8e22-b9b956eeb26f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.218642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 47/98] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:44.224558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533b84b-080c-459d-be61-ea225ae8727e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.225169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533b84b-080c-459d-be61-ea225ae8727e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.225546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533b84b-080c-459d-be61-ea225ae8727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.245055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-e533b84b-080c-459d-be61-ea225ae8727e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:44.246905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e533b84b-080c-459d-be61-ea225ae8727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:44.323017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334336-b979-476c-bc2e-d2e1abf48be2 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.323349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334336-b979-476c-bc2e-d2e1abf48be2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.323770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334336-b979-476c-bc2e-d2e1abf48be2 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.325903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 48/99] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:44.328518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 52/100] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:44.340819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c914f63-d732-4b05-8bab-a6cb99cf09a7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.341103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c914f63-d732-4b05-8bab-a6cb99cf09a7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.341382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c914f63-d732-4b05-8bab-a6cb99cf09a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.366402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7c914f63-d732-4b05-8bab-a6cb99cf09a7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:44.367625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7c914f63-d732-4b05-8bab-a6cb99cf09a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:44.383133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533b84b-080c-459d-be61-ea225ae8727e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.424125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:44.510313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 49/101] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:44.539873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c914f63-d732-4b05-8bab-a6cb99cf09a7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.579965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533b84b-080c-459d-be61-ea225ae8727e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.639861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7cf618-9683-4375-80af-1db520828433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.640354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:44.640671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cf618-9683-4375-80af-1db520828433 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.640985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cf618-9683-4375-80af-1db520828433 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.641243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7cf618-9683-4375-80af-1db520828433 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.653812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7cf618-9683-4375-80af-1db520828433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-1f7cf618-9683-4375-80af-1db520828433 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:44.663041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:44.663963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:44.664760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 53/102] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:06:44.666148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368a864-257b-4214-a1d6-a6e32addabe0 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.666148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368a864-257b-4214-a1d6-a6e32addabe0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.666148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368a864-257b-4214-a1d6-a6e32addabe0 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.672038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 54/103] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:44.673368 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c64cb-8d72-4ba3-ae57-16b395739109 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.673667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c64cb-8d72-4ba3-ae57-16b395739109 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.673953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c64cb-8d72-4ba3-ae57-16b395739109 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.675308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 55/104] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:44.676047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c64cb-8d72-4ba3-ae57-16b395739109 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.681466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c914f63-d732-4b05-8bab-a6cb99cf09a7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.745089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab47aa-f146-4fed-b0a8-b5c19c318a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.746040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab47aa-f146-4fed-b0a8-b5c19c318a5d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.746576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab47aa-f146-4fed-b0a8-b5c19c318a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.747045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab47aa-f146-4fed-b0a8-b5c19c318a5d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.757831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab47aa-f146-4fed-b0a8-b5c19c318a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-63ab47aa-f146-4fed-b0a8-b5c19c318a5d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:44.766416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:44.767246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:06:44.768029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:44.770334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 50/105] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:06:44.771940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b7aa-dbd5-46d8-9750-6a8a00f2d149 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.772427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b7aa-dbd5-46d8-9750-6a8a00f2d149 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.772906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41b7aa-dbd5-46d8-9750-6a8a00f2d149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.790715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ad41b7aa-dbd5-46d8-9750-6a8a00f2d149 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:44.792246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ad41b7aa-dbd5-46d8-9750-6a8a00f2d149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:44.800534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:44.822344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c64cb-8d72-4ba3-ae57-16b395739109 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:44.860846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:44.861794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.862258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.862678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.863635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:44.864109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:44.864308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 51/106] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:44.865819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988379d6-b5a8-4a26-9717-7f69cd9fc58b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:44.866109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:44.866259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988379d6-b5a8-4a26-9717-7f69cd9fc58b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:44.866630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988379d6-b5a8-4a26-9717-7f69cd9fc58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:44.876531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 56/107] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:44.877614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.883991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-988379d6-b5a8-4a26-9717-7f69cd9fc58b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:44.885333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-988379d6-b5a8-4a26-9717-7f69cd9fc58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:44.945711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:44.957960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 52/108] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:44.959133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988379d6-b5a8-4a26-9717-7f69cd9fc58b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:44.997311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4852826-e87f-4e8e-ad3f-d666ff15b523 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.034643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.035340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.035633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.035960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.036848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:45.037123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.038517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.048420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 57/109] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:45.049554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.058053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.081852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988379d6-b5a8-4a26-9717-7f69cd9fc58b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.120360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.122255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.122738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.123216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.124734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.126397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.128356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 53/110] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:44 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:45.129538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.175479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a9e75-4cd0-43f0-99cc-f230c68805b8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.193912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.214364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.215990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.216099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.216349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.217553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.219337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.221024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 58/111] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:45.222034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.241720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f689e3-8361-4202-815c-99818328f0d1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.280426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.281608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.282235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.282684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.294517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:45.298733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:45.303893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:45.304917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:45.305708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:45.306269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 54/112] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:45.307208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.336036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b029c-e8cd-4301-9aec-95e3db518070 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.375567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.376472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.376758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.377037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] PATH_INFO: `/v3/users/5f766bfecd75454d86786f22b4fdb9ab` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.391631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] RBAC: Authorizing `identity:get_user(user_id=5f766bfecd75454d86786f22b4fdb9ab)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.393142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.395014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 59/113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:45 2022] GET /identity/v3/users/5f766bfecd75454d86786f22b4fdb9ab => generated 289 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:45.399918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.419225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b73bf0-ee74-4f8a-aaa4-6ee2e463d286 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.440270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.461087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98954b27-a638-49c8-ac0c-e7040d2a9e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.461976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954b27-a638-49c8-ac0c-e7040d2a9e6a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.490202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954b27-a638-49c8-ac0c-e7040d2a9e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.490202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98954b27-a638-49c8-ac0c-e7040d2a9e6a None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.490202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98954b27-a638-49c8-ac0c-e7040d2a9e6a None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.490202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98954b27-a638-49c8-ac0c-e7040d2a9e6a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.490202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 55/114] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:45 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:45.490202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53aaa5-62a4-4a12-a89f-9cf98571e2d7 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.490659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53aaa5-62a4-4a12-a89f-9cf98571e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.490659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53aaa5-62a4-4a12-a89f-9cf98571e2d7 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.490659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 56/115] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:45.490659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53aaa5-62a4-4a12-a89f-9cf98571e2d7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.523443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49454627-560d-41fe-86b2-b30022f95121 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.554588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.561440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.562127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.562416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.562696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.563661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.565324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.589556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53aaa5-62a4-4a12-a89f-9cf98571e2d7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.624382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.625032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.625326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.625657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.626415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:45.626762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.628495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.637365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 57/116] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:45.638265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.656042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 60/117] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:06:45 2022] POST /identity/v3/projects => generated 302 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:45.665361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:45.693242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.753647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe9a86-b533-4b39-a014-52770b52f2af None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.792337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d72396-eb4c-47c6-a3f4-48ee4025654b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.793047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d72396-eb4c-47c6-a3f4-48ee4025654b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.793339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d72396-eb4c-47c6-a3f4-48ee4025654b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.793623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d72396-eb4c-47c6-a3f4-48ee4025654b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.794485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24ccfe-39c5-4155-9f4d-437b9ca7436c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:45.835325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d8268-a5a2-4882-88a5-df3ccbd96e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:45.836175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8268-a5a2-4882-88a5-df3ccbd96e59 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.836464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8268-a5a2-4882-88a5-df3ccbd96e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.836745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8268-a5a2-4882-88a5-df3ccbd96e59 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/591c45f7bb064291b85f4b4a3998abe6/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.838805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d8268-a5a2-4882-88a5-df3ccbd96e59 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cdddc85fc3414a8028b384e27eabad, user_id=591c45f7bb064291b85f4b4a3998abe6, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.841010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d8268-a5a2-4882-88a5-df3ccbd96e59 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.870160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d72396-eb4c-47c6-a3f4-48ee4025654b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:45.872288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d72396-eb4c-47c6-a3f4-48ee4025654b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:45.917949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 61/118] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:06:45 2022] PUT /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/591c45f7bb064291b85f4b4a3998abe6/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:45.919874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596786-e84e-4ac2-91fc-d124c9774d21 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.920166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596786-e84e-4ac2-91fc-d124c9774d21 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.920445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c596786-e84e-4ac2-91fc-d124c9774d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:45.927827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.939956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5c596786-e84e-4ac2-91fc-d124c9774d21 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:45.941304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5c596786-e84e-4ac2-91fc-d124c9774d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:45.991212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:45.997039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 58/119] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:45.999090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eae29-8537-4cc1-a85a-78fbc2644592 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:45.999090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eae29-8537-4cc1-a85a-78fbc2644592 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:45.999296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096eae29-8537-4cc1-a85a-78fbc2644592 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.004723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 59/120] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:46.004723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eae29-8537-4cc1-a85a-78fbc2644592 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.046020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 62/121] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:46.047353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596786-e84e-4ac2-91fc-d124c9774d21 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.125035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096eae29-8537-4cc1-a85a-78fbc2644592 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:46.165617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c596786-e84e-4ac2-91fc-d124c9774d21 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:46.166126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36271e-89a9-4373-af71-b5eda82b4abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:46.167291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36271e-89a9-4373-af71-b5eda82b4abb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.167586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36271e-89a9-4373-af71-b5eda82b4abb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.167997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36271e-89a9-4373-af71-b5eda82b4abb None admin] PATH_INFO: `/v3/users/ef098ca75a4b4231b612fc38df12339d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.170639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36271e-89a9-4373-af71-b5eda82b4abb None admin] RBAC: Authorizing `identity:get_user(user_id=ef098ca75a4b4231b612fc38df12339d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:46.172484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36271e-89a9-4373-af71-b5eda82b4abb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:46.173964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 60/122] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/users/ef098ca75a4b4231b612fc38df12339d => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:46.177394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9483fb-78b1-41fc-8062-fe7abfca30b0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.177394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9483fb-78b1-41fc-8062-fe7abfca30b0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.177394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9483fb-78b1-41fc-8062-fe7abfca30b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.195120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8f9483fb-78b1-41fc-8062-fe7abfca30b0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:46.196971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8f9483fb-78b1-41fc-8062-fe7abfca30b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:46.210275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:46.219794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:46.220813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.221103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.221379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.223027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:46.225208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:46.227649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 63/123] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:46.229039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.277096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 61/124] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:46.278146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9483fb-78b1-41fc-8062-fe7abfca30b0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.292117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:46.363823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67831df5-a40b-497e-8414-a86b5c2adbb8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:46.403046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:46.403798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.404086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.404455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.408259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9483fb-78b1-41fc-8062-fe7abfca30b0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:46.416606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:46.426223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:46.427080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:46.428079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 64/125] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:46.429444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.446778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:46.447809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.448208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.448556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.450026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:46.451590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:46.453492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 62/126] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:46.454366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.478357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:46.563278 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:46.566962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698652d-a62f-4954-947b-f63689b6ff51 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:46.596784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b80667f-2003-4940-bf32-c028345cb682 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:46.614286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:46.615047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.615349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.615629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.616726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] RBAC: Adding query filter params (role.id=c8e1e9ae479b42ddb805138503822330, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=5f766bfecd75454d86786f22b4fdb9ab) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:46.617055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:46.618871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:46.629075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 65/127] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/role_assignments?user.id=5f766bfecd75454d86786f22b4fdb9ab&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=c8e1e9ae479b42ddb805138503822330 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:46.631954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.647874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:46.649055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.649570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.650043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.673433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:46.682980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:46.684521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:46.685305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:46.704848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 63/128] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:46.706157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.809932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebd4fc-1def-401e-85fa-b67ac20a16da None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:46.850781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:46.851623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:46.852027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:46.852842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:46.853488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:46.859776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:46.859776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:46.866620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 66/129] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:46.867860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:46.900841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:46.909276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:46.950818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54959c-8b32-4805-9dfa-dc89d9de4cf9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:47.006186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:47.007242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:47.007651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:47.008026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/ef098ca75a4b4231b612fc38df12339d/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:47.010514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cdddc85fc3414a8028b384e27eabad, user_id=ef098ca75a4b4231b612fc38df12339d, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:47.012691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:47.039355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56884f8-67ca-44c9-b75c-4acfdd1af426 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:47.078163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:47.078923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:47.079276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:47.079650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:47.080505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:47.080885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:47.082459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:47.092789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 67/130] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:47.093891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:47.111511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 64/131] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:06:46 2022] PUT /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/ef098ca75a4b4231b612fc38df12339d/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:47.112506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:47.256320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:47.264610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af2000-d910-4c4d-a2df-ae9f731137dd None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:47.305544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:47.307174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:47.307469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:47.307779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:47.309790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:47.311413 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:47.313342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 68/132] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:47 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:47.320047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:47.422073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe0a64-7aae-4b66-8d35-601fdee92db8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:47.493911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5938c-9a53-486b-b29f-29c288672a25 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:47.512340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:47.512340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:47.512340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:47.512340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:47.513266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:47.514826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:47.516928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 65/133] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:47 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:47.522344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:47.536112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:47.572496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:47.573861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:47.574914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:47.575392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:47.576983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:47.578704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:47.580646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 69/134] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:47 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:47.586559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:47.653179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:47.795646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3d01-2733-4d68-adf8-da13081d4ccd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:47.869437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:47.870323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:47.870610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:47.870882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] PATH_INFO: `/v3/projects/c515f5d4ee914043984885a852baa240` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:47.872208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] RBAC: Authorizing `identity:get_project(project_id=c515f5d4ee914043984885a852baa240)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:47.873702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:47.879870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 66/135] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:47 2022] GET /identity/v3/projects/c515f5d4ee914043984885a852baa240 => generated 312 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:47.882192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8c84f-4442-44b0-91d5-fc8ffa3fdab1 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:47.882476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:47.976494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e1567-4ddd-4706-936c-c4015cccda13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:47.977416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e1567-4ddd-4706-936c-c4015cccda13 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:47.977710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e1567-4ddd-4706-936c-c4015cccda13 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:47.977994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e1567-4ddd-4706-936c-c4015cccda13 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/591c45f7bb064291b85f4b4a3998abe6/roles/c23de713167345639a2085add0a54ae6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:47.984520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e1567-4ddd-4706-936c-c4015cccda13 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cdddc85fc3414a8028b384e27eabad, user_id=591c45f7bb064291b85f4b4a3998abe6, role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:47.985984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e1567-4ddd-4706-936c-c4015cccda13 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:48.017463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:48.065511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30410-d672-4aed-8858-8d9a587f8fff None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:48.078018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 70/136] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:06:47 2022] PUT /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/591c45f7bb064291b85f4b4a3998abe6/roles/c23de713167345639a2085add0a54ae6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:06:48.113655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bae3c2-3183-4824-bd1b-f3cca3c1defd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:48.114574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bae3c2-3183-4824-bd1b-f3cca3c1defd None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.114861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bae3c2-3183-4824-bd1b-f3cca3c1defd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.115139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bae3c2-3183-4824-bd1b-f3cca3c1defd None admin] PATH_INFO: `/v3/projects/c515f5d4ee914043984885a852baa240/users/591c45f7bb064291b85f4b4a3998abe6/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.117238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bae3c2-3183-4824-bd1b-f3cca3c1defd None admin] RBAC: Authorizing `identity:create_grant(project_id=c515f5d4ee914043984885a852baa240, user_id=591c45f7bb064291b85f4b4a3998abe6, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:48.118932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bae3c2-3183-4824-bd1b-f3cca3c1defd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:48.221604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 67/137] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:06:47 2022] PUT /identity/v3/projects/c515f5d4ee914043984885a852baa240/users/591c45f7bb064291b85f4b4a3998abe6/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:06:48.425429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5944c7-a29e-4beb-8636-31a453d182f8 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.425780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5944c7-a29e-4beb-8636-31a453d182f8 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.426002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5944c7-a29e-4beb-8636-31a453d182f8 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.427486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 71/138] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:48.434512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925616ee-2b54-4949-aa51-e2821e92af14 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.435017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925616ee-2b54-4949-aa51-e2821e92af14 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.435088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925616ee-2b54-4949-aa51-e2821e92af14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.457918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-925616ee-2b54-4949-aa51-e2821e92af14 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:48.459235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-925616ee-2b54-4949-aa51-e2821e92af14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:48.556868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 68/139] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:48.600179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5944c7-a29e-4beb-8636-31a453d182f8 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:48.729519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:48.738747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5a443-6b6b-4409-9e58-412164747830 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.746336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5a443-6b6b-4409-9e58-412164747830 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.746336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5a443-6b6b-4409-9e58-412164747830 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.746336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 69/140] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:48.747534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0c11b-f408-444f-aab6-adfe2e033f61 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.747850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0c11b-f408-444f-aab6-adfe2e033f61 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.748129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0c11b-f408-444f-aab6-adfe2e033f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.757166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5944c7-a29e-4beb-8636-31a453d182f8 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:48.784042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-33e0c11b-f408-444f-aab6-adfe2e033f61 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:48.790082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-33e0c11b-f408-444f-aab6-adfe2e033f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:48.830081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:48.830904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:48.831199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:48.831476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:48.836543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:48.838030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:48.839720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 72/141] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:48.849774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:48.894615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:48.918046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 70/142] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:48.960046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0c11b-f408-444f-aab6-adfe2e033f61 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:49.044123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7960f-1bc2-467f-800e-cdef116fd7b7 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:49.099055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:49.099773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.100087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.100485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:49.101242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:49.102726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:49.156044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0c11b-f408-444f-aab6-adfe2e033f61 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:49.201536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:49.202343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.202627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.202958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:49.219818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:49.227618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:49.228538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:49.229297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 71/143] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:06:49.235579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:49.258682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 73/144] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:06:48 2022] POST /identity/v3/users => generated 283 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:49.306999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:49.383917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff6019-dd18-4659-ae8a-360500f38759 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:49.441372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:49.442189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.442449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.442730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:49.443508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:49.443840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:49.445278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:49.461072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 72/145] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:49.466987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:49.484279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:49.592021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1f76b-f785-49cf-9b6c-837bd6b12892 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:49.628700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:49.629593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.629877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.630155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:49.631415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:49.632972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:49.634638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 74/146] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:49 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:49.641444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:49.774594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0526d-c935-4153-9448-2c1b270d1d07 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:49.792150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:49.811728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bbda7-a6e9-443e-9864-f05418de0bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:49.812437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bbda7-a6e9-443e-9864-f05418de0bff None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:49.812731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bbda7-a6e9-443e-9864-f05418de0bff None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:49.813011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bbda7-a6e9-443e-9864-f05418de0bff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:49.889024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bbda7-a6e9-443e-9864-f05418de0bff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:49.890527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bbda7-a6e9-443e-9864-f05418de0bff None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:50.002018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 73/147] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:50.007687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:50.079267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30520faf-67f5-412f-9da9-32645d1543bb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.080054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30520faf-67f5-412f-9da9-32645d1543bb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.080662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30520faf-67f5-412f-9da9-32645d1543bb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.082652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 74/148] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:50.088738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acbaaa-7149-4e3c-99e4-057e6337e603 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.089439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acbaaa-7149-4e3c-99e4-057e6337e603 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.090475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acbaaa-7149-4e3c-99e4-057e6337e603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.114968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-70acbaaa-7149-4e3c-99e4-057e6337e603 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:50.116717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-70acbaaa-7149-4e3c-99e4-057e6337e603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:50.119237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:50.137862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:50.172966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0770853-4cb4-47ff-9385-da098d3b0de9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:50.238284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc6a13-f8d4-428c-89be-5c0c1dfa8714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:50.239536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc6a13-f8d4-428c-89be-5c0c1dfa8714 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.240213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc6a13-f8d4-428c-89be-5c0c1dfa8714 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.240687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc6a13-f8d4-428c-89be-5c0c1dfa8714 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.244645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc6a13-f8d4-428c-89be-5c0c1dfa8714 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:50.246712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bc6a13-f8d4-428c-89be-5c0c1dfa8714 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:50.248731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 75/149] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:50 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:50.256060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833879b-7af7-4683-aaf8-4007ac43e050 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.256361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833879b-7af7-4683-aaf8-4007ac43e050 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.256646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833879b-7af7-4683-aaf8-4007ac43e050 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.258263 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 76/150] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:50.258981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833879b-7af7-4683-aaf8-4007ac43e050 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:50.272149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 75/151] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:50.273769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a75c9f-dc26-4cba-8685-0543f5508162 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.274260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a75c9f-dc26-4cba-8685-0543f5508162 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.281315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a75c9f-dc26-4cba-8685-0543f5508162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.316809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c8a75c9f-dc26-4cba-8685-0543f5508162 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:50.318222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c8a75c9f-dc26-4cba-8685-0543f5508162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:50.449736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 76/152] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:50.451467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a75c9f-dc26-4cba-8685-0543f5508162 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:50.457862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833879b-7af7-4683-aaf8-4007ac43e050 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:50.583343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:50.606179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:50.606863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.607155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.607434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.608319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=591c45f7bb064291b85f4b4a3998abe6) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:50.608633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:50.610072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:50.628670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a75c9f-dc26-4cba-8685-0543f5508162 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:50.638440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 77/153] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:50 2022] GET /identity/v3/role_assignments?user.id=591c45f7bb064291b85f4b4a3998abe6&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=1183a502796d46e580a92e3c20f04103 => generated 608 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:50.639307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:50.666011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:50.710056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:50.717533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:50.717863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:50.718150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:50.748729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:50.763223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:50.764182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:06:50.764936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:50.769408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 77/154] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:50.777007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:50.995531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450877e-9d39-4ca1-8db7-5acd958efc31 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:51.049328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:51.051629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15767f13-4bd8-4e13-94f0-8460c1152d1f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:51.066529 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:51.067560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:51.068102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:51.068592 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:51.069555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:51.070353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:51.072341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:51.092574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 78/155] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:51.094585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:51.180498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:51.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:51.181611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:51.181889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:51.207245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:51.215877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:51.216843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:51.217624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:51.221671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 78/156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 578 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:51.224536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:51.287118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c54d29-dc04-4e30-b75e-59805b2a5cc3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:51.349625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:51.350633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:51.350921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:51.351199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] PATH_INFO: `/v3/users/5f766bfecd75454d86786f22b4fdb9ab` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:51.354611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] RBAC: Authorizing `identity:get_user(user_id=5f766bfecd75454d86786f22b4fdb9ab)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:51.356700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:51.358708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 79/157] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:51 2022] GET /identity/v3/users/5f766bfecd75454d86786f22b4fdb9ab => generated 289 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:51.364000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:51.373095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:51.463543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006032e-9601-4e4e-971c-614ff9d14574 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:51.483549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:51.560713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:51.561334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:51.561704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:51.561890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed643954-a6f1-4438-8648-74ed6f7858c0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:51.561989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:51.562665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:51.562973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:51.564390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:51.576892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 79/158] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:51.582723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:51.629699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:51.630608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:51.631009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:51.635196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:51.637238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:51.638738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:51.640956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 80/159] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:51 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:51.647428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:51.799780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bcfeda-94ed-402a-aa85-b27a3b84a265 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:51.834059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abe169c-046b-48a6-90a1-3121656cea46 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:51.836680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:51.844490 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5d371-ff8f-48a9-a820-dd529229200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:51.852987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5d371-ff8f-48a9-a820-dd529229200b None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:51.852987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5d371-ff8f-48a9-a820-dd529229200b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:51.852987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5d371-ff8f-48a9-a820-dd529229200b None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/5f766bfecd75454d86786f22b4fdb9ab/roles/c8e1e9ae479b42ddb805138503822330` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:51.852987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5d371-ff8f-48a9-a820-dd529229200b None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cdddc85fc3414a8028b384e27eabad, user_id=5f766bfecd75454d86786f22b4fdb9ab, role_id=c8e1e9ae479b42ddb805138503822330)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:51.852987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5d371-ff8f-48a9-a820-dd529229200b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:51.875621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:51.881528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:51.882490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:51.882859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:51.883196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] PATH_INFO: `/v3/users/ef098ca75a4b4231b612fc38df12339d` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:51.884481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] RBAC: Authorizing `identity:get_user(user_id=ef098ca75a4b4231b612fc38df12339d)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:51.886013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:51.888544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 80/160] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:51 2022] GET /identity/v3/users/ef098ca75a4b4231b612fc38df12339d => generated 247 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:51.894485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:51.929135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 81/161] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:06:51 2022] PUT /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad/users/5f766bfecd75454d86786f22b4fdb9ab/roles/c8e1e9ae479b42ddb805138503822330 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:06:52.018736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b70c52-f6e5-4494-b5fc-09eeef2588bb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:52.040211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fee5d4-4bc2-4d3f-b3c2-780579e54ccf None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:52.040516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fee5d4-4bc2-4d3f-b3c2-780579e54ccf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:52.040775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fee5d4-4bc2-4d3f-b3c2-780579e54ccf None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:52.042277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 82/162] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:06:52.048558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98910a6-c1da-4d11-be23-4df9a85b93bb None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:52.048835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98910a6-c1da-4d11-be23-4df9a85b93bb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:52.049116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98910a6-c1da-4d11-be23-4df9a85b93bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:52.064396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:52.065116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:52.065389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:52.065712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:52.080561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c98910a6-c1da-4d11-be23-4df9a85b93bb None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:52.081825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c98910a6-c1da-4d11-be23-4df9a85b93bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:52.164221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:52.166861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:52.169188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:52.252383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 83/163] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:52.253789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e04a9-8117-4ee8-a266-493bb64ab7f3 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:52.254081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e04a9-8117-4ee8-a266-493bb64ab7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:52.254362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986e04a9-8117-4ee8-a266-493bb64ab7f3 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:52.255927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 84/164] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:52.263188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71c28f-f516-4f84-9092-f27f40bd9af0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:52.263476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71c28f-f516-4f84-9092-f27f40bd9af0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:52.263804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a71c28f-f516-4f84-9092-f27f40bd9af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:52.295610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7a71c28f-f516-4f84-9092-f27f40bd9af0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:52.296831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7a71c28f-f516-4f84-9092-f27f40bd9af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:52.394348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 85/165] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 26 21:06:52.395201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 81/166] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:52.399677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71c28f-f516-4f84-9092-f27f40bd9af0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:52.417372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:52.482416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:52.627553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a71c28f-f516-4f84-9092-f27f40bd9af0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:52.653835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:52.664518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e470f73f-aed0-48d5-bb07-227b90c0c2a4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:52.739192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:52.740102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:52.740383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:52.740486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:52.740712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:52.741273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:52.741545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:52.741829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:52.744041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:52.745591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:52.748219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 82/167] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:52 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:52.749004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:52.764303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:52.780758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:52.781844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:52.782627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 86/168] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:06:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:52.783257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:52.963120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2111-7abc-4166-989f-6989a3236bf2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:52.982575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e1da0-32cb-42d1-80f8-650753416a71 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:52.994280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:53.057730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.058428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.058720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.058999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.059873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=ef098ca75a4b4231b612fc38df12339d) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:53.060191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.061666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.067482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.072373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.072667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.072952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.080841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 87/169] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:52 2022] GET /identity/v3/role_assignments?user.id=ef098ca75a4b4231b612fc38df12339d&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:53.081700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.105013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:53.117165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:53.118082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:53.118872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 83/170] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:53.127629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.197759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:53.270112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7dd59f-db61-4459-b3e2-d485193c078d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:53.323183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.323921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.324238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.324518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.325277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:53.325589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.326995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.337135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 88/171] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:06:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:53.347258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.393247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:53.404878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377d526-ae43-4d9a-a5df-7ccbcfd9908e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:53.472311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e28a7-890d-4fba-8027-26eb2000a4e2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:53.486379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.487335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.487783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.488201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.489100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:53.489551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.491076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.511761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.511978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 84/172] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:53.513115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.513537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.513985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.515438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.517225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.517452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.519605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 89/173] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:53 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:53.526524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.663636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053c990-799e-469e-8b98-72e97738f830 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:53.667128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:53.707716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:53.721035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb445f5-1556-42a7-80cf-1b95356abaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.721980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb445f5-1556-42a7-80cf-1b95356abaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.722404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb445f5-1556-42a7-80cf-1b95356abaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.722861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb445f5-1556-42a7-80cf-1b95356abaa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.739423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e6db4-6f9d-4b65-9473-ea03263ff33d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:53.794958 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077994f-4eb9-442e-be30-401974582562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:53.795788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077994f-4eb9-442e-be30-401974582562 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.796106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077994f-4eb9-442e-be30-401974582562 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.796343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077994f-4eb9-442e-be30-401974582562 None admin] PATH_INFO: `/v3/users/591c45f7bb064291b85f4b4a3998abe6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.797570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077994f-4eb9-442e-be30-401974582562 None admin] RBAC: Authorizing `identity:get_user(user_id=591c45f7bb064291b85f4b4a3998abe6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.799266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077994f-4eb9-442e-be30-401974582562 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.801420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 85/174] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:53 2022] GET /identity/v3/users/591c45f7bb064291b85f4b4a3998abe6 => generated 275 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:53.808824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077994f-4eb9-442e-be30-401974582562 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.809436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb445f5-1556-42a7-80cf-1b95356abaa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:53.811156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb445f5-1556-42a7-80cf-1b95356abaa3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:53.934583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 90/175] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 21:06:53.937211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08fd6-a04b-4ba5-a7e5-d1d6f44268ff None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.937507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08fd6-a04b-4ba5-a7e5-d1d6f44268ff None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.937789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b08fd6-a04b-4ba5-a7e5-d1d6f44268ff None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.939181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 91/176] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:53.940343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd2462-7515-4293-b618-0809aaca1117 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:53.940631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd2462-7515-4293-b618-0809aaca1117 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:53.940914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd2462-7515-4293-b618-0809aaca1117 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:53.946826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 92/177] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:53.947692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd2462-7515-4293-b618-0809aaca1117 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:53.968115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:53.979406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077994f-4eb9-442e-be30-401974582562 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.026497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0a633d-de23-4a53-9e48-82c0637ae736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.027903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a633d-de23-4a53-9e48-82c0637ae736 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.028283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a633d-de23-4a53-9e48-82c0637ae736 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.028672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0a633d-de23-4a53-9e48-82c0637ae736 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.056751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.069063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd2462-7515-4293-b618-0809aaca1117 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.110562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb942a-3733-49ed-9e91-5dd7c4fe8fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.112129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb942a-3733-49ed-9e91-5dd7c4fe8fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.113199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb942a-3733-49ed-9e91-5dd7c4fe8fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.113685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb942a-3733-49ed-9e91-5dd7c4fe8fe0 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.114892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a633d-de23-4a53-9e48-82c0637ae736 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.115523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb942a-3733-49ed-9e91-5dd7c4fe8fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.116561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0a633d-de23-4a53-9e48-82c0637ae736 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.117346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb942a-3733-49ed-9e91-5dd7c4fe8fe0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.121648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 93/178] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:53 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:54.123133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a699b-343e-47ad-8c4c-650f9ca9ef8a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.123596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a699b-343e-47ad-8c4c-650f9ca9ef8a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.124047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a699b-343e-47ad-8c4c-650f9ca9ef8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.148358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-852a699b-343e-47ad-8c4c-650f9ca9ef8a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:54.150550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-852a699b-343e-47ad-8c4c-650f9ca9ef8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:54.234282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 86/179] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:54.235927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d28d1-c1b3-4384-b7e5-6ca7341a101a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.236319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d28d1-c1b3-4384-b7e5-6ca7341a101a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.236690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1d28d1-c1b3-4384-b7e5-6ca7341a101a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.259778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7e1d28d1-c1b3-4384-b7e5-6ca7341a101a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:54.259778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7e1d28d1-c1b3-4384-b7e5-6ca7341a101a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:54.260056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 94/180] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:54.262298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a699b-343e-47ad-8c4c-650f9ca9ef8a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.266803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.329875 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.355990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 87/181] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:54.355990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d28d1-c1b3-4384-b7e5-6ca7341a101a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.401916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852a699b-343e-47ad-8c4c-650f9ca9ef8a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.440192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.441205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.441434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.441857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.442825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] RBAC: Adding query filter params (role.id=c23de713167345639a2085add0a54ae6, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=591c45f7bb064291b85f4b4a3998abe6) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:54.443140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.445331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.455344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 95/182] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/role_assignments?user.id=591c45f7bb064291b85f4b4a3998abe6&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=c23de713167345639a2085add0a54ae6 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:54.456388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.479043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1d28d1-c1b3-4384-b7e5-6ca7341a101a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.530795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eee0d0-c79c-4e9f-894d-006d0667729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.532479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eee0d0-c79c-4e9f-894d-006d0667729b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.533005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eee0d0-c79c-4e9f-894d-006d0667729b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.533592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eee0d0-c79c-4e9f-894d-006d0667729b None admin] PATH_INFO: `/v3/projects/c515f5d4ee914043984885a852baa240` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.535274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eee0d0-c79c-4e9f-894d-006d0667729b None admin] RBAC: Authorizing `identity:get_project(project_id=c515f5d4ee914043984885a852baa240)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.537041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eee0d0-c79c-4e9f-894d-006d0667729b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.539201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 88/183] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/projects/c515f5d4ee914043984885a852baa240 => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:54.541052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288af15-263d-454f-9e1e-6a872d3713b0 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.541542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288af15-263d-454f-9e1e-6a872d3713b0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.542029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288af15-263d-454f-9e1e-6a872d3713b0 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.543687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 89/184] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:54.544689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288af15-263d-454f-9e1e-6a872d3713b0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.556122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.606067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2519cc-0ff2-45c3-ade1-6eebcf9b7110 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.624160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.648329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fd6d7-422d-4a9b-9fc0-c2ba143cff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.649567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd6d7-422d-4a9b-9fc0-c2ba143cff50 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.650045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd6d7-422d-4a9b-9fc0-c2ba143cff50 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.650509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fd6d7-422d-4a9b-9fc0-c2ba143cff50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.664261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fd6d7-422d-4a9b-9fc0-c2ba143cff50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-011fd6d7-422d-4a9b-9fc0-c2ba143cff50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:54.673464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:54.674385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:54.675895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 96/185] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:54.676561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b19f8-7c6b-45d4-a48c-db42d6765551 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.676625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b19f8-7c6b-45d4-a48c-db42d6765551 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.676895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b19f8-7c6b-45d4-a48c-db42d6765551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.691848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288af15-263d-454f-9e1e-6a872d3713b0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.702135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-907b19f8-7c6b-45d4-a48c-db42d6765551 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:54.703677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-907b19f8-7c6b-45d4-a48c-db42d6765551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:54.752267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.753082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.753365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.753650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.767390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:54.776665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:54.777625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:54.778514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 90/186] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:54.784078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.794669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 97/187] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:54.795626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b19f8-7c6b-45d4-a48c-db42d6765551 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.857724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.904169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b19f8-7c6b-45d4-a48c-db42d6765551 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.908061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5672698-880e-417f-bea0-3568c10388f9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:54.921578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:54.940723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.941414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.941792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.942078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.942842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:54.943152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.943266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:54.943935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:54.944223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:54.944499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:54.944664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.945354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=c515f5d4ee914043984885a852baa240, user.id=591c45f7bb064291b85f4b4a3998abe6) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:54.945669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:54.947425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:54.955012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 91/188] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/role_assignments?user.id=591c45f7bb064291b85f4b4a3998abe6&scope.project.id=c515f5d4ee914043984885a852baa240&role.id=1183a502796d46e580a92e3c20f04103 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:54.955946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:54.956629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 98/189] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:54.957643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:55.088005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68c20e-f770-4d0c-a02d-d10683482799 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.104516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb26a38-70a3-4f6a-9742-c6f7b992000b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.125801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:55.126977 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.127488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.127793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.128074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.128852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:55.129164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:55.130633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:55.141011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 92/190] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:55.142046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:55.148215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.149298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.149724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.150187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.152745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:55.154404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:55.156284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 99/191] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:55.157196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:55.188760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:55.252173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d70e3-bba3-4835-8315-dcd320151f34 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.270507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bad5ae-92a7-4e3a-b0b6-449a07ce5603 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.286094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa5bfb-dc45-47e4-92da-4d756ea471e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.286965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa5bfb-dc45-47e4-92da-4d756ea471e4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.287297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa5bfb-dc45-47e4-92da-4d756ea471e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.287573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa5bfb-dc45-47e4-92da-4d756ea471e4 None admin] PATH_INFO: `/v3/users/92da47b4830848a989964351669df6f9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.300567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa5bfb-dc45-47e4-92da-4d756ea471e4 None admin] RBAC: Authorizing `identity:get_user(user_id=92da47b4830848a989964351669df6f9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:55.302013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa5bfb-dc45-47e4-92da-4d756ea471e4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:55.303381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.303658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 93/192] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:55 2022] GET /identity/v3/users/92da47b4830848a989964351669df6f9 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:55.304286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.304566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.304841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] PATH_INFO: `/v3/users/92da47b4830848a989964351669df6f9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.306044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] RBAC: Authorizing `identity:get_user(user_id=92da47b4830848a989964351669df6f9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:55.307488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:55.308996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882672-46aa-4ba6-b68c-f20267d2a2c3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.309171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 100/193] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:55 2022] GET /identity/v3/users/92da47b4830848a989964351669df6f9 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:55.309280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882672-46aa-4ba6-b68c-f20267d2a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.309549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf882672-46aa-4ba6-b68c-f20267d2a2c3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.309904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:55.311008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 94/194] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:55.312529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf882672-46aa-4ba6-b68c-f20267d2a2c3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:55.366467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:55.412237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:55.412554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e6a40-c68d-4457-8bb9-27cfb5debb6f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.417119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf882672-46aa-4ba6-b68c-f20267d2a2c3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.445203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.445868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.446151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.446694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.447405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:55.448939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:55.451145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64370757-125f-44e7-872f-4098e8707ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.451816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370757-125f-44e7-872f-4098e8707ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.452099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370757-125f-44e7-872f-4098e8707ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.452371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64370757-125f-44e7-872f-4098e8707ea5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.520138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64370757-125f-44e7-872f-4098e8707ea5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:55.521719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64370757-125f-44e7-872f-4098e8707ea5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:55.540467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 101/195] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:06:55 2022] POST /identity/v3/users => generated 297 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:06:55.541593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:55.636327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:55.641038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 95/196] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:55.642795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac291a3-b343-4b0e-8e71-f67d5918ee5d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.643366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac291a3-b343-4b0e-8e71-f67d5918ee5d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.643838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac291a3-b343-4b0e-8e71-f67d5918ee5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.721785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7ac291a3-b343-4b0e-8e71-f67d5918ee5d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:55.722999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7ac291a3-b343-4b0e-8e71-f67d5918ee5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:55.745402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2d698-6fe2-4523-8872-869e3e036acc None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.758590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:55.787790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.788451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.788741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:55.789017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:55.805481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 96/197] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:55.806441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac291a3-b343-4b0e-8e71-f67d5918ee5d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:55.882773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:55.884445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:55.945908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac291a3-b343-4b0e-8e71-f67d5918ee5d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:55.987149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:55.997443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:55.998316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:55.998596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.000148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:56.000372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:56.002001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:56.003914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 97/198] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:55 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:56.004681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:56.007445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 102/199] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:56.009683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:56.023715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:56.114796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176ee9-d991-4f5f-956d-2db5f540e7dd None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:56.137180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd3a73-b263-437a-bca8-e4e5d6e4a76a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:56.162013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:56.163466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:56.164066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.164349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:56.177275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:56.178066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:56.178305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:56.178404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.178772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:56.179668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] RBAC: Adding query filter params (role.id=c23de713167345639a2085add0a54ae6, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=92da47b4830848a989964351669df6f9) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:56.180076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:56.181621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:56.187443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:56.188732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:06:56.190954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:56.192423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 103/200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:56 2022] GET /identity/v3/role_assignments?user.id=92da47b4830848a989964351669df6f9&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=c23de713167345639a2085add0a54ae6 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:56.192570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 98/201] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:56.192703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:56.196857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:56.328271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:56.356192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:56.389459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a852b3-74ed-4a0b-bed1-4078c988558f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:56.435224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:56.436595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:56.436889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.437169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:56.438621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:56.440418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:56.442687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 104/202] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:56 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:56.448827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:56.450240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3bcd8-f30d-4011-97c5-ac96b4e7a04f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:56.521179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:56.522362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:56.522849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.523325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:56.524312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:56.524920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:56.526617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:56.548897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 99/203] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:56.554523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:56.565366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354023f4-0ca6-4e36-8c42-61e6eb0b62ac None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:56.606282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:56.606865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:56.607152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.607436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:56.608557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=92da47b4830848a989964351669df6f9) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:56.608931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:56.610842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:56.617612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 105/204] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:56 2022] GET /identity/v3/role_assignments?user.id=92da47b4830848a989964351669df6f9&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:56.766102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac1eb2-631a-46a0-964d-74377090f846 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:56.790652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:56.822360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:56.824311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:56.824311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.824311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] PATH_INFO: `/v3/users/5f766bfecd75454d86786f22b4fdb9ab` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:56.828522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] RBAC: Authorizing `identity:get_user(user_id=5f766bfecd75454d86786f22b4fdb9ab)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:56.830006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:56.831906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 100/205] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:56 2022] GET /identity/v3/users/5f766bfecd75454d86786f22b4fdb9ab => generated 289 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:56.837509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:56.850657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:56.956214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf6072-a168-4af6-8368-216f2f37c6a3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:56.992235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:56.993417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:56.993918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:56.994459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:57.067764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:57.069581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:57.080667 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:57.178020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 106/206] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:57.183198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:57.344596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7d2ce8-9d50-4939-956d-3ca8c3f33023 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:57.393794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc97ea-19f9-41cd-bf58-665596d1b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:57.394660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc97ea-19f9-41cd-bf58-665596d1b753 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:57.394943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc97ea-19f9-41cd-bf58-665596d1b753 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:57.395216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc97ea-19f9-41cd-bf58-665596d1b753 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:57.396427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc97ea-19f9-41cd-bf58-665596d1b753 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:57.397872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc97ea-19f9-41cd-bf58-665596d1b753 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:57.399880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 101/207] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:57 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:57.408147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:57.545077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789a4f8-32d7-43b3-ada6-3102eca936c2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:57.567767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:57.592750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da052809-0621-464e-9b41-3d191407298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:57.593983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052809-0621-464e-9b41-3d191407298e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:57.594821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052809-0621-464e-9b41-3d191407298e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:57.595608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da052809-0621-464e-9b41-3d191407298e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:57.597554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da052809-0621-464e-9b41-3d191407298e None admin] RBAC: Adding query filter params (role.id=c8e1e9ae479b42ddb805138503822330, scope.project.id=a2cdddc85fc3414a8028b384e27eabad, user.id=5f766bfecd75454d86786f22b4fdb9ab) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:57.598112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da052809-0621-464e-9b41-3d191407298e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:57.599910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da052809-0621-464e-9b41-3d191407298e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:57.607154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 107/208] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:57 2022] GET /identity/v3/role_assignments?user.id=5f766bfecd75454d86786f22b4fdb9ab&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=c8e1e9ae479b42ddb805138503822330 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:58.259523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a131a-3afa-457b-ad72-3ae48739bb27 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.260589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a131a-3afa-457b-ad72-3ae48739bb27 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.261082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a131a-3afa-457b-ad72-3ae48739bb27 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.263494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 102/209] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:58.268720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781fadd-76b9-4265-b9b8-5b3d9cbe1a90 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.269600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781fadd-76b9-4265-b9b8-5b3d9cbe1a90 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.270088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781fadd-76b9-4265-b9b8-5b3d9cbe1a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.293890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d781fadd-76b9-4265-b9b8-5b3d9cbe1a90 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:58.295437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d781fadd-76b9-4265-b9b8-5b3d9cbe1a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:58.368248 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 108/210] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:58.392457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a131a-3afa-457b-ad72-3ae48739bb27 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.413496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84394e1-b7e4-4bea-a12f-ab5500b48eb4 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.413819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84394e1-b7e4-4bea-a12f-ab5500b48eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.414068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84394e1-b7e4-4bea-a12f-ab5500b48eb4 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.415612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 109/211] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:06:58.421777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a15692-fa12-42e9-b227-60cb1116a348 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.422282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a15692-fa12-42e9-b227-60cb1116a348 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.422832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a15692-fa12-42e9-b227-60cb1116a348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.441112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-10a15692-fa12-42e9-b227-60cb1116a348 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:58.442338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-10a15692-fa12-42e9-b227-60cb1116a348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:58.457158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:58.513706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a131a-3afa-457b-ad72-3ae48739bb27 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.525850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 110/212] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:06:58.554373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.555800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a15692-fa12-42e9-b227-60cb1116a348 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.556751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.557038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.557370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.559456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:58.561123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:58.563124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 103/213] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:58.569179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.600123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:58.683256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4e1f1-5457-42b8-b455-356cc6aeabe1 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.685068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a15692-fa12-42e9-b227-60cb1116a348 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.711440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:58.724010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9e2f0c-a464-4e78-bcf0-9afbe766a66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.724694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2f0c-a464-4e78-bcf0-9afbe766a66f None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.724984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2f0c-a464-4e78-bcf0-9afbe766a66f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.725266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e2f0c-a464-4e78-bcf0-9afbe766a66f None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.725684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59809fe2-335e-43c2-8142-e4a0ef7dfff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.725986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2f0c-a464-4e78-bcf0-9afbe766a66f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:58.726487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809fe2-335e-43c2-8142-e4a0ef7dfff0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.726773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809fe2-335e-43c2-8142-e4a0ef7dfff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.727093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59809fe2-335e-43c2-8142-e4a0ef7dfff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.727609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9e2f0c-a464-4e78-bcf0-9afbe766a66f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:58.738473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59809fe2-335e-43c2-8142-e4a0ef7dfff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-59809fe2-335e-43c2-8142-e4a0ef7dfff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:58.746405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:58.754770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 111/214] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:06:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:58.755671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69d8a-2cb7-41ad-add3-f0ec4d2609a8 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.756420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69d8a-2cb7-41ad-add3-f0ec4d2609a8 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.756420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a69d8a-2cb7-41ad-add3-f0ec4d2609a8 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.756420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 112/215] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:58.756420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69d8a-2cb7-41ad-add3-f0ec4d2609a8 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.838387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 104/216] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/users => generated 297 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:06:58.840043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4143d5e-7351-49d8-b907-5f16e7784e74 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.840525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4143d5e-7351-49d8-b907-5f16e7784e74 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.840809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4143d5e-7351-49d8-b907-5f16e7784e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.859681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f4143d5e-7351-49d8-b907-5f16e7784e74 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:58.861060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f4143d5e-7351-49d8-b907-5f16e7784e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:58.866090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:58.883405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a69d8a-2cb7-41ad-add3-f0ec4d2609a8 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:58.938686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:58.939377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:58.939662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:58.939960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:58.940729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] RBAC: Adding query filter params (name=member) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:58.941041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:58.942494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:58.953183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 113/217] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:06:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:06:58.958224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 105/218] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:58.962725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:58.982024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4143d5e-7351-49d8-b907-5f16e7784e74 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.103833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff8f9e-2058-456a-b70e-ef79b7986a95 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.107935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4143d5e-7351-49d8-b907-5f16e7784e74 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.123389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.161994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.162347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.162528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.162801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] PATH_INFO: `/v3/users/f87d1f98920642a59bcb07e00baa12da` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.165032 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.165846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.169660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.169660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.179838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] RBAC: Authorizing `identity:get_user(user_id=f87d1f98920642a59bcb07e00baa12da)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.180564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:59.181831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.184733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 114/219] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:58 2022] GET /identity/v3/users/f87d1f98920642a59bcb07e00baa12da => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:59.192670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:59.196479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:59.197379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:59.198154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 106/220] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:06:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:06:59.207030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.215495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.324956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065e07c-9a92-4194-b835-aa725d57b121 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.335140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456952da-83fb-4433-a167-1619787d0c20 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.379951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94408dba-3827-47d1-a316-69ef75fe7d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.380543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408dba-3827-47d1-a316-69ef75fe7d91 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.380876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408dba-3827-47d1-a316-69ef75fe7d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.381159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94408dba-3827-47d1-a316-69ef75fe7d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.382210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2e6e21-160a-4947-b100-e1fb5f58b115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.382886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e6e21-160a-4947-b100-e1fb5f58b115 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.383170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e6e21-160a-4947-b100-e1fb5f58b115 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.383448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2e6e21-160a-4947-b100-e1fb5f58b115 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.384511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e6e21-160a-4947-b100-e1fb5f58b115 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:06:59.385724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e6e21-160a-4947-b100-e1fb5f58b115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.387860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2e6e21-160a-4947-b100-e1fb5f58b115 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.399051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 107/221] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:59.400772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9aec8-f7e3-4e2e-b9a5-46befa2ec3da None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.401064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9aec8-f7e3-4e2e-b9a5-46befa2ec3da None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.401342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9aec8-f7e3-4e2e-b9a5-46befa2ec3da None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.402942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 108/222] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:06:59.408165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9aec8-f7e3-4e2e-b9a5-46befa2ec3da None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.448143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.459393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94408dba-3827-47d1-a316-69ef75fe7d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.461192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94408dba-3827-47d1-a316-69ef75fe7d91 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.480214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.545493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9aec8-f7e3-4e2e-b9a5-46befa2ec3da None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.575106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 115/223] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:06:59.576943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee8f1a-3325-4378-9faa-c717333120ad None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.577431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee8f1a-3325-4378-9faa-c717333120ad None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.577900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ee8f1a-3325-4378-9faa-c717333120ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.588918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.590778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.591077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.591370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] PATH_INFO: `/v3/users/92da47b4830848a989964351669df6f9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.592957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] RBAC: Authorizing `identity:get_user(user_id=92da47b4830848a989964351669df6f9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.594423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.596147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 109/224] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/users/92da47b4830848a989964351669df6f9 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:59.597053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.597802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a7ee8f1a-3325-4378-9faa-c717333120ad None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:06:59.599259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a7ee8f1a-3325-4378-9faa-c717333120ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:06:59.673601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 116/225] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:06:59.674862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee8f1a-3325-4378-9faa-c717333120ad None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.752660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.756918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1235256-aad5-4445-8134-2de37efbdc5c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.764114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:06:59.798624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.800056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.800516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.800971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.802365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.803994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.805915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 110/226] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:59.806846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.827966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ee8f1a-3325-4378-9faa-c717333120ad None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.864616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.865483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.865773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.866055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.867401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:06:59.869039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:06:59.871873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 117/227] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:06:59.872791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.913228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8bff4-c478-49ef-bf0f-43b9495dd7e5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:06:59.950344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:06:59.951501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:06:59.952017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:06:59.952497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:06:59.963462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:06:59.971555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:06:59.972489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:06:59.973301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:06:59.975767 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 111/228] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:06:59.976738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:06:59.987989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757dc9cd-8e97-4356-87c8-66511ed92779 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:00.000095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:00.015341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:00.030398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:00.031826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:00.032318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:00.032816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:00.034215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=f87d1f98920642a59bcb07e00baa12da) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:00.035065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:00.037040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:00.044572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 118/229] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:06:59 2022] GET /identity/v3/role_assignments?user.id=f87d1f98920642a59bcb07e00baa12da&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=1183a502796d46e580a92e3c20f04103 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:00.045591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:00.161731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a21418-2a71-4f73-afcc-949f4c9d85b2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:00.208345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9942670d-b6fe-49da-bcc7-43d793365ca0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:00.239944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:00.240874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:00.241163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:00.241448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/92da47b4830848a989964351669df6f9/roles/c23de713167345639a2085add0a54ae6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:00.243897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dad1ade88dd4d0b8c70347b618248c9, user_id=92da47b4830848a989964351669df6f9, role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:00.245518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:00.265133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:00.265810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:00.266092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:00.266362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:00.267113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:00.276043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:00.276043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:00.277793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 119/230] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:00.283784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:00.325478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:00.338576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 112/231] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:06:59 2022] PUT /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/92da47b4830848a989964351669df6f9/roles/c23de713167345639a2085add0a54ae6 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:00.402034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5007f-d15f-43db-bb10-f3517c93acff None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:00.442265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:00.443533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:00.444071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:00.444563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] PATH_INFO: `/v3/users/92da47b4830848a989964351669df6f9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:00.446054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] RBAC: Authorizing `identity:get_user(user_id=92da47b4830848a989964351669df6f9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:00.448078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:00.450443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 120/232] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:00 2022] GET /identity/v3/users/92da47b4830848a989964351669df6f9 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:00.459978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:00.601070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:00.617375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96b8adf-2284-4530-b376-87f066814981 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:00.654487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5023e023-0676-4142-af8f-11695769b81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:00.655369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023e023-0676-4142-af8f-11695769b81f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:00.655654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023e023-0676-4142-af8f-11695769b81f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:00.655965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023e023-0676-4142-af8f-11695769b81f None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:00.657166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023e023-0676-4142-af8f-11695769b81f None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:00.658789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5023e023-0676-4142-af8f-11695769b81f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:00.660597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 113/233] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:00 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:00.666099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:00.799913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0392e332-966a-46ad-bdfc-9c6e31bd3e4f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:00.801770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:00.845418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc22b1-9a19-44d0-88bc-2600c99454d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:00.846697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc22b1-9a19-44d0-88bc-2600c99454d1 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:00.847184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc22b1-9a19-44d0-88bc-2600c99454d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:00.847668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc22b1-9a19-44d0-88bc-2600c99454d1 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/92da47b4830848a989964351669df6f9/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:00.849896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc22b1-9a19-44d0-88bc-2600c99454d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dad1ade88dd4d0b8c70347b618248c9, user_id=92da47b4830848a989964351669df6f9, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:00.851570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc22b1-9a19-44d0-88bc-2600c99454d1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:00.952162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 121/234] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:00 2022] PUT /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/92da47b4830848a989964351669df6f9/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:01.551362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871176-e863-4814-b872-75fb4ab383e1 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:01.551933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871176-e863-4814-b872-75fb4ab383e1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:01.551933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d871176-e863-4814-b872-75fb4ab383e1 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:01.553576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 114/235] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:01.559332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c83ce-7382-41b1-9106-e577e3d7860e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:01.559776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c83ce-7382-41b1-9106-e577e3d7860e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:01.559940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c83ce-7382-41b1-9106-e577e3d7860e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:01.578320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a12c83ce-7382-41b1-9106-e577e3d7860e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:01.579524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a12c83ce-7382-41b1-9106-e577e3d7860e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:01.586839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8830e-11ef-4562-b98a-7dc62a42eaa2 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:01.587183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8830e-11ef-4562-b98a-7dc62a42eaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:01.587464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8830e-11ef-4562-b98a-7dc62a42eaa2 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:01.588877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 115/236] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:01.605869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0113c-8f6e-4cd7-921a-2f2dafec8b24 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:01.606175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0113c-8f6e-4cd7-921a-2f2dafec8b24 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:01.606431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0113c-8f6e-4cd7-921a-2f2dafec8b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:01.632002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-dfa0113c-8f6e-4cd7-921a-2f2dafec8b24 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:01.641170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dfa0113c-8f6e-4cd7-921a-2f2dafec8b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:01.702528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 122/237] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:01.735801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c83ce-7382-41b1-9106-e577e3d7860e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:01.779172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 116/238] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:01.807060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0113c-8f6e-4cd7-921a-2f2dafec8b24 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:01.860826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:01.869821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12c83ce-7382-41b1-9106-e577e3d7860e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:01.922600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:01.923408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:01.923691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:01.923994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:01.926265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:01.928559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:01.933998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 123/239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:01.937110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:01.981610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:02.010075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0113c-8f6e-4cd7-921a-2f2dafec8b24 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:02.070879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:02.071921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.072205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.072478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.089991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:02.099147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:02.100121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:02.100917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:02.102016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 117/240] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:02.110565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:02.128178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae200e07-bd15-4608-b147-4a54f2f00a88 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:02.188803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:02.189487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.189772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.190048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.190772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:02.193439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:02.266112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69b72f-458a-4158-8e4d-001e912ebe4b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:02.300273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 124/241] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:07:01 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:02.317331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:02.318037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.318323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.318701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.319665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:02.320020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:02.321850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:02.336209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 118/242] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:02.342780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:02.376426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:02.510986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e58cc-847b-42eb-a158-a56f45d05798 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:02.557714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a63af-876b-48d6-a7d5-acde38b20154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:02.559145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a63af-876b-48d6-a7d5-acde38b20154 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.559430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a63af-876b-48d6-a7d5-acde38b20154 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.559986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a63af-876b-48d6-a7d5-acde38b20154 None admin] PATH_INFO: `/v3/users/0a08c5eda47f41d8a1d635e60b73216f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.579783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a63af-876b-48d6-a7d5-acde38b20154 None admin] RBAC: Authorizing `identity:get_user(user_id=0a08c5eda47f41d8a1d635e60b73216f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:02.581221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a63af-876b-48d6-a7d5-acde38b20154 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:02.583309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 125/243] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:02 2022] GET /identity/v3/users/0a08c5eda47f41d8a1d635e60b73216f => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:02.588862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:02.600975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:02.621636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37031e6-3a95-4692-b561-8a3b12eec32f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.622258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37031e6-3a95-4692-b561-8a3b12eec32f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.622538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37031e6-3a95-4692-b561-8a3b12eec32f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.624156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 126/244] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:02.634659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccfccc-51b0-4322-8383-2fc0ac3b45e2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.635219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccfccc-51b0-4322-8383-2fc0ac3b45e2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.635825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ccfccc-51b0-4322-8383-2fc0ac3b45e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.655735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-22ccfccc-51b0-4322-8383-2fc0ac3b45e2 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:02.657473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-22ccfccc-51b0-4322-8383-2fc0ac3b45e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:02.736210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15207f17-46d1-4771-a46c-55b48bbf7296 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:02.760434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 127/245] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:02.784981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccfccc-51b0-4322-8383-2fc0ac3b45e2 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:02.785605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:02.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.786698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.786980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.793690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:02.876659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:02.878885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:02.889702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:02.918934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ccfccc-51b0-4322-8383-2fc0ac3b45e2 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:02.962023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:02.963223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:02.963876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:02.964383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:02.982512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:02.992700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:02.993620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:02.994399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 128/246] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:03.001211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:03.014162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 119/247] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:03.021537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:03.085474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:03.135052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bce03-74ed-416c-a846-67b7d4b7a972 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:03.142523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaad54-28b9-43e2-a513-966ebd30ffee None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:03.154042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:03.177730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:03.178833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.179367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.179908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.180874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:03.181414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:03.183132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:03.186820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:03.188107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.188403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.188685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.189947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:03.191399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:03.193707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 120/248] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:03 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:03.199924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:03.201315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 129/249] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:03.209779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:03.324282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047bef2-4225-43df-8e57-b95c6c5641c0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:03.328612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b34e1-8fea-405d-98ee-35b9af746ddb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:03.353450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:03.365466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f1b5b-8044-45fd-aef1-84b8425e3f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:03.369847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f1b5b-8044-45fd-aef1-84b8425e3f71 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.369847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f1b5b-8044-45fd-aef1-84b8425e3f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.369847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f1b5b-8044-45fd-aef1-84b8425e3f71 None admin] PATH_INFO: `/v3/users/f87d1f98920642a59bcb07e00baa12da` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.369847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f1b5b-8044-45fd-aef1-84b8425e3f71 None admin] RBAC: Authorizing `identity:get_user(user_id=f87d1f98920642a59bcb07e00baa12da)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:03.371456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f1b5b-8044-45fd-aef1-84b8425e3f71 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:03.373549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 130/250] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:03 2022] GET /identity/v3/users/f87d1f98920642a59bcb07e00baa12da => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:03.374986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b47ab6-3c93-483d-9496-85d54c64bc25 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.379768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b47ab6-3c93-483d-9496-85d54c64bc25 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.380482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b47ab6-3c93-483d-9496-85d54c64bc25 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.382455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 131/251] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:03.382890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af0106-c15e-43d6-8133-b5b7cfae6e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:03.383559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af0106-c15e-43d6-8133-b5b7cfae6e1b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.383992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af0106-c15e-43d6-8133-b5b7cfae6e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.384164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af0106-c15e-43d6-8133-b5b7cfae6e1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.385111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af0106-c15e-43d6-8133-b5b7cfae6e1b None admin] RBAC: Adding query filter params (role.id=c8e1e9ae479b42ddb805138503822330, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=0a08c5eda47f41d8a1d635e60b73216f) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:03.385386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af0106-c15e-43d6-8133-b5b7cfae6e1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:03.385751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b47ab6-3c93-483d-9496-85d54c64bc25 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:03.387580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af0106-c15e-43d6-8133-b5b7cfae6e1b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:03.401167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 121/252] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:03 2022] GET /identity/v3/role_assignments?user.id=0a08c5eda47f41d8a1d635e60b73216f&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=c8e1e9ae479b42ddb805138503822330 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:03.402673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66497a0f-eec1-49da-96a8-62b1d8284b93 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.402963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66497a0f-eec1-49da-96a8-62b1d8284b93 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.403239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66497a0f-eec1-49da-96a8-62b1d8284b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.443129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-66497a0f-eec1-49da-96a8-62b1d8284b93 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:03.445595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-66497a0f-eec1-49da-96a8-62b1d8284b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:03.537458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:03.557364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b47ab6-3c93-483d-9496-85d54c64bc25 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:03.603922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:03.606505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.607032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.607577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.609425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:03.611124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:03.613288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 132/253] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:03 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:03.620816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:03.661337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 122/254] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:03.697463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66497a0f-eec1-49da-96a8-62b1d8284b93 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:03.754190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:03.800793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6a28e-a1a7-4c0d-a47e-b156b5e5d9eb None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:03.842712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:03.843772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.844063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.844337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/f87d1f98920642a59bcb07e00baa12da/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:03.847084 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] RBAC: Authorizing `identity:create_grant(project_id=2dad1ade88dd4d0b8c70347b618248c9, user_id=f87d1f98920642a59bcb07e00baa12da, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:03.848734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:03.920550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66497a0f-eec1-49da-96a8-62b1d8284b93 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:03.939856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 133/255] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:03 2022] PUT /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/f87d1f98920642a59bcb07e00baa12da/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:07:03.982791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064fd122-69c3-46a3-a2d5-13447697dd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:03.983620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fd122-69c3-46a3-a2d5-13447697dd44 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:03.983936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fd122-69c3-46a3-a2d5-13447697dd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:03.984213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fd122-69c3-46a3-a2d5-13447697dd44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:04.004288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064fd122-69c3-46a3-a2d5-13447697dd44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-064fd122-69c3-46a3-a2d5-13447697dd44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:04.012880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:04.013821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:04.014619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 123/256] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:04.020309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:04.104979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e40f0-9626-4b95-8875-9da1c55f24df None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:04.105842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e40f0-9626-4b95-8875-9da1c55f24df None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:04.106109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e40f0-9626-4b95-8875-9da1c55f24df None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:04.107489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 124/257] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:04.113089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da33d0-ca64-4096-96f4-7f0998b2b2f7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:04.113381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da33d0-ca64-4096-96f4-7f0998b2b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:04.113663 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da33d0-ca64-4096-96f4-7f0998b2b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:04.152909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c9da33d0-ca64-4096-96f4-7f0998b2b2f7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:04.154216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c9da33d0-ca64-4096-96f4-7f0998b2b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:04.204113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:04.291381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e47d0-9b7b-4dd4-8114-ea73b662f5aa None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:04.321758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:04.335302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:04.336338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:04.337170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:04.337965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:04.339139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:04.339677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:04.341313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:04.355905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 134/258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:04.357356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 125/259] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:04.362796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:04.382906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da33d0-ca64-4096-96f4-7f0998b2b2f7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:04.570364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd1aec1-ab43-41f0-8816-db64c6125c1b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:04.595359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da33d0-ca64-4096-96f4-7f0998b2b2f7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:04.632408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:04.633292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:04.633572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:04.633844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] PATH_INFO: `/v3/users/92da47b4830848a989964351669df6f9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:04.635459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] RBAC: Authorizing `identity:get_user(user_id=92da47b4830848a989964351669df6f9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:04.646830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:04.646830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 135/260] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:04 2022] GET /identity/v3/users/92da47b4830848a989964351669df6f9 => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:04.653550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:04.687799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:04.688614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:04.688896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:04.689232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:04.726756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:04.734975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:04.735899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:04.736714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:04.747808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 126/261] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:07:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:04.751620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:04.776131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:04.795225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:04.832545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3772b9-c7d3-42d5-a266-d93b56205d09 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:04.882597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:04.883257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:04.883539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:04.883865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:04.943146 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63bbca-9c64-47cf-b0e5-f58c0740471e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:05.016102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:05.016780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:05.017061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:05.017334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:05.018094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:05.018398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:05.021124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:05.022593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:05.025649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:05.044086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 127/262] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:05.049571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:05.160204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:05.173260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:05.202254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220548-c2d7-45c6-a72c-c1a8a0340dbc None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:05.254605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:05.255533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:05.255850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:05.256132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] PATH_INFO: `/v3/users/92da47b4830848a989964351669df6f9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:05.266464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] RBAC: Authorizing `identity:get_user(user_id=92da47b4830848a989964351669df6f9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:05.268056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:05.270220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 128/263] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:05 2022] GET /identity/v3/users/92da47b4830848a989964351669df6f9 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:05.275696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:05.309380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 136/264] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:05.314924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:05.448232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf476c0d-d80d-45d1-b999-1171477bff15 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:05.485990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da44ead-1cad-4ef5-b653-5e426f4e71fa None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:05.524529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:05.525574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:05.526089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:05.526567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:05.533401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:05.534284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:05.534571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:05.534845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:05.543781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:05.543781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:05.544127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 137/265] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:05 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:05.549735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:05.563438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:05.580407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:05.662374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254a07c-902b-4f2f-8ed1-21f1da59a54e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:05.710189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:05.712356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:05.720601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafdb33-f6ca-4f2d-b947-7906e24c0a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:05.721289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafdb33-f6ca-4f2d-b947-7906e24c0a33 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:05.721576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafdb33-f6ca-4f2d-b947-7906e24c0a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:05.721913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafdb33-f6ca-4f2d-b947-7906e24c0a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:05.722801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafdb33-f6ca-4f2d-b947-7906e24c0a33 None admin] RBAC: Adding query filter params (role.id=c23de713167345639a2085add0a54ae6, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=92da47b4830848a989964351669df6f9) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:05.723124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafdb33-f6ca-4f2d-b947-7906e24c0a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:05.728507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafdb33-f6ca-4f2d-b947-7906e24c0a33 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:05.739288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 138/266] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:05 2022] GET /identity/v3/role_assignments?user.id=92da47b4830848a989964351669df6f9&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=c23de713167345639a2085add0a54ae6 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:05.864939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ac07-a59c-4294-9d32-691ac567436f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:05.865595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ac07-a59c-4294-9d32-691ac567436f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:05.866068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8ac07-a59c-4294-9d32-691ac567436f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:05.867683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 139/267] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:05.870539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 129/268] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 21:07:05.875868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:05.876190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42696-a165-423d-8a7e-1f2e27c3dd35 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:05.876720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42696-a165-423d-8a7e-1f2e27c3dd35 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:05.877199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a42696-a165-423d-8a7e-1f2e27c3dd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:05.907703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-82a42696-a165-423d-8a7e-1f2e27c3dd35 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:05.909303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-82a42696-a165-423d-8a7e-1f2e27c3dd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:05.956407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:06.067407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 140/269] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:06.103081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1f637-101d-4d25-8a7f-106f94a583dd None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.115497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42696-a165-423d-8a7e-1f2e27c3dd35 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.143658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:06.171019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.171926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.172261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.172545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.174242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:06.177731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:06.182694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 130/270] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:05 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:06.185444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.284414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a42696-a165-423d-8a7e-1f2e27c3dd35 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.320925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2767e9-0102-4c5e-b47e-2253e08340f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.322226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2767e9-0102-4c5e-b47e-2253e08340f0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.322654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2767e9-0102-4c5e-b47e-2253e08340f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.323068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2767e9-0102-4c5e-b47e-2253e08340f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.325834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2767e9-0102-4c5e-b47e-2253e08340f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:06.327466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2767e9-0102-4c5e-b47e-2253e08340f0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:06.329364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 141/271] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:06.330741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76acbf-5fab-4026-b343-003803ab7a4e None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.331174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76acbf-5fab-4026-b343-003803ab7a4e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.331613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76acbf-5fab-4026-b343-003803ab7a4e None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.337265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 142/272] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:06.338586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76acbf-5fab-4026-b343-003803ab7a4e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.362011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f306f-7b31-4283-9a11-8349e3a4fb12 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.388701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:06.407156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fde4e0e-6f62-4eec-866c-d68759c3bd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.407982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4e0e-6f62-4eec-866c-d68759c3bd25 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.408263 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4e0e-6f62-4eec-866c-d68759c3bd25 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.408544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fde4e0e-6f62-4eec-866c-d68759c3bd25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.409501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde4e0e-6f62-4eec-866c-d68759c3bd25 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=92da47b4830848a989964351669df6f9) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:06.409823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde4e0e-6f62-4eec-866c-d68759c3bd25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:06.411284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fde4e0e-6f62-4eec-866c-d68759c3bd25 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:06.418485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 131/273] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/role_assignments?user.id=92da47b4830848a989964351669df6f9&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:06.420328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147edde-e537-44b9-bd59-55438318cc4e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.420642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147edde-e537-44b9-bd59-55438318cc4e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.420930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147edde-e537-44b9-bd59-55438318cc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.432113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:06.456089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76acbf-5fab-4026-b343-003803ab7a4e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.456835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f147edde-e537-44b9-bd59-55438318cc4e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:06.458273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f147edde-e537-44b9-bd59-55438318cc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:06.496854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8196a1-7f02-433b-bc75-b24a017e1771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.497846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8196a1-7f02-433b-bc75-b24a017e1771 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.498308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8196a1-7f02-433b-bc75-b24a017e1771 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.498734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8196a1-7f02-433b-bc75-b24a017e1771 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.499685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8196a1-7f02-433b-bc75-b24a017e1771 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:06.501460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8196a1-7f02-433b-bc75-b24a017e1771 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:06.568081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 132/274] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:06.595949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147edde-e537-44b9-bd59-55438318cc4e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.622214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 143/275] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:07:06 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:06.710980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147edde-e537-44b9-bd59-55438318cc4e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.726252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:06.753498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.754324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.754611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.754887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.768229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:06.771102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c6af5-f755-415e-b8fb-cede7c4a2ba4 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.773913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c6af5-f755-415e-b8fb-cede7c4a2ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.773913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c6af5-f755-415e-b8fb-cede7c4a2ba4 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.773913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 144/276] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:06.776773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:06.778298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:06.779519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 133/277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:06.784901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.785655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3ff5-204c-406c-b803-78f0e19fb0b6 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.786223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3ff5-204c-406c-b803-78f0e19fb0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.786691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e3ff5-204c-406c-b803-78f0e19fb0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.805208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-370e3ff5-204c-406c-b803-78f0e19fb0b6 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:06.806866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-370e3ff5-204c-406c-b803-78f0e19fb0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:06.891932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 145/278] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:06.902448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465bce23-852b-4587-9b87-72950204ec67 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:06.913889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3ff5-204c-406c-b803-78f0e19fb0b6 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.939274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:06.939994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:06.940280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:06.940579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:06.941378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:06.941683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:06.943084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:06.952887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 134/279] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:06.958948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:06.992126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:06.993212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:07.035435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e3ff5-204c-406c-b803-78f0e19fb0b6 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.068009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3304ea-86bb-4862-8ea8-334324ddd2c5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.076261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.077497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.077995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.078550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.093419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:07.101920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:07.102930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:07.103728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:07.107493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 146/280] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:07.110318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.111207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.111491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.111859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] PATH_INFO: `/v3/users/7ade02842f6d411ba7215a9f1a76b950` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.113584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.137516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] RBAC: Authorizing `identity:get_user(user_id=7ade02842f6d411ba7215a9f1a76b950)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:07.140147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.142786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 135/281] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:06 2022] GET /identity/v3/users/7ade02842f6d411ba7215a9f1a76b950 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:07.150947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.248676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd516b9-0c7e-40a8-a80a-d780efea5807 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.256183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d6e04-9f0e-4fb1-9661-fd0211f3a7fd None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.295795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac950995-a2b9-4793-81df-b611bd3e45df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.295795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac950995-a2b9-4793-81df-b611bd3e45df None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.295795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac950995-a2b9-4793-81df-b611bd3e45df None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.295795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac950995-a2b9-4793-81df-b611bd3e45df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.296224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac950995-a2b9-4793-81df-b611bd3e45df None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, user.id=7ade02842f6d411ba7215a9f1a76b950, scope.system=all) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:07.296224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac950995-a2b9-4793-81df-b611bd3e45df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:07.296224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac950995-a2b9-4793-81df-b611bd3e45df None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.301185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.301658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 136/282] 10.222.0.42 () {56 vars in 1308 bytes} [Thu May 26 21:07:07 2022] GET /identity/v3/role_assignments?user.id=7ade02842f6d411ba7215a9f1a76b950&scope.system=all&role.id=1183a502796d46e580a92e3c20f04103 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:07.302382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.302868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.303342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.304285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:07.310480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:07.310480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.316885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 147/283] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:07.322838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877f9f-e8ea-46b9-9cb7-c85865d74aa5 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.322838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877f9f-e8ea-46b9-9cb7-c85865d74aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.322838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877f9f-e8ea-46b9-9cb7-c85865d74aa5 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.322838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 137/284] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:07.324712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.328285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28836-507e-4741-9bfd-b9024621c8ad None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.328569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28836-507e-4741-9bfd-b9024621c8ad None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.328846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c28836-507e-4741-9bfd-b9024621c8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.359871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f5c28836-507e-4741-9bfd-b9024621c8ad None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:07.361105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f5c28836-507e-4741-9bfd-b9024621c8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:07.364899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:07.392133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:07.497054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 138/285] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:07.499249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84c58d-a1a6-4aea-8c69-6e7a7a1c18a1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.549994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c28836-507e-4741-9bfd-b9024621c8ad None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.552991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.553891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.554172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.554440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] PATH_INFO: `/v3/users/0a08c5eda47f41d8a1d635e60b73216f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.561625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] RBAC: Authorizing `identity:get_user(user_id=0a08c5eda47f41d8a1d635e60b73216f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:07.563122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.565062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 148/286] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:07 2022] GET /identity/v3/users/0a08c5eda47f41d8a1d635e60b73216f => generated 297 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:07.570953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.652177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:07.700078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab272368-0d13-4b42-b0c7-657b09845f27 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.726382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:07.753568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.754364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.754649 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.754924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.757315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c28836-507e-4741-9bfd-b9024621c8ad None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.759655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:07.761189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.763494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 149/287] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:07 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:07.769357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.803985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.804792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.805124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.805405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.817096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:07.826110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:07.827035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:07.828275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 139/288] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:07.835477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:07.893242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634056a4-244c-4e0a-b00b-0119d4ff9984 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:07.943370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:07.974128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:07.975030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:07.975318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:07.975595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/0a08c5eda47f41d8a1d635e60b73216f/roles/c8e1e9ae479b42ddb805138503822330` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:07.978722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] RBAC: Authorizing `identity:create_grant(project_id=2dad1ade88dd4d0b8c70347b618248c9, user_id=0a08c5eda47f41d8a1d635e60b73216f, role_id=c8e1e9ae479b42ddb805138503822330)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:07.981516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:07.984417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b64e9b-7b1f-48c3-a6bb-aaac8cb2bdf9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:08.027733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:08.028733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:08.029173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:08.029770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:08.030689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:08.031204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:08.033009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:08.043541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 140/289] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:08.049064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:08.060315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:08.062412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 150/290] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:07 2022] PUT /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/users/0a08c5eda47f41d8a1d635e60b73216f/roles/c8e1e9ae479b42ddb805138503822330 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:08.164654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5fd51-3df3-45d9-a48f-b7b3f24cecae None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:08.203656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:08.204824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:08.205261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:08.205708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] PATH_INFO: `/v3/users/f87d1f98920642a59bcb07e00baa12da` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:08.207267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] RBAC: Authorizing `identity:get_user(user_id=f87d1f98920642a59bcb07e00baa12da)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:08.208917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:08.211067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 141/291] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:08 2022] GET /identity/v3/users/f87d1f98920642a59bcb07e00baa12da => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:08.216670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:08.362608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e23cb9-fbbd-4ccf-99f5-d352b45f8e9b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:08.399723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:08.400411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:08.400695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:08.400975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:08.459514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:08.474205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:08.475682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:08.578467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 151/292] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:08.584116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:08.716303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:08.736609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2397d617-9370-46f8-8633-aa198b165272 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:08.787006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d55f97-a15b-4281-9064-7627c4a66cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:08.787911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d55f97-a15b-4281-9064-7627c4a66cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:08.788198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d55f97-a15b-4281-9064-7627c4a66cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:08.788476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d55f97-a15b-4281-9064-7627c4a66cd6 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:08.789697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d55f97-a15b-4281-9064-7627c4a66cd6 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:08.791169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d55f97-a15b-4281-9064-7627c4a66cd6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:08.792933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 142/293] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:08 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:08.798791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:08.920149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:08.939599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da4adb-ed75-43aa-b42c-c288281a6abe None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:08.986462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4af10-a797-4956-88bf-b4cba5b2aa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:08.987460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4af10-a797-4956-88bf-b4cba5b2aa02 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:08.987997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4af10-a797-4956-88bf-b4cba5b2aa02 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:08.988465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4af10-a797-4956-88bf-b4cba5b2aa02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:08.989511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4af10-a797-4956-88bf-b4cba5b2aa02 None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=f87d1f98920642a59bcb07e00baa12da) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:08.990025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4af10-a797-4956-88bf-b4cba5b2aa02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:08.991677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4af10-a797-4956-88bf-b4cba5b2aa02 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:09.002852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 152/294] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:08 2022] GET /identity/v3/role_assignments?user.id=f87d1f98920642a59bcb07e00baa12da&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=1183a502796d46e580a92e3c20f04103 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:09.490503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97527a-0e01-4611-8d7b-bf5e75b29fa3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.490901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97527a-0e01-4611-8d7b-bf5e75b29fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.491154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa97527a-0e01-4611-8d7b-bf5e75b29fa3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.492882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 143/295] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:09.500619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50fd73-367b-4958-9ddf-b31c72386cd3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.500906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50fd73-367b-4958-9ddf-b31c72386cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.501173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50fd73-367b-4958-9ddf-b31c72386cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.523401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-dc50fd73-367b-4958-9ddf-b31c72386cd3 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:09.524736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-dc50fd73-367b-4958-9ddf-b31c72386cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:09.606605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 153/296] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:09.632228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97527a-0e01-4611-8d7b-bf5e75b29fa3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:09.759241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa97527a-0e01-4611-8d7b-bf5e75b29fa3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:09.767178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa706d-7704-43c3-8eb5-4357e1a1a2ba None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.767467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa706d-7704-43c3-8eb5-4357e1a1a2ba None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.767767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa706d-7704-43c3-8eb5-4357e1a1a2ba None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.769279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 154/297] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:09.775725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78353775-339a-48b6-88fc-1d6b4252ec3c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.776042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78353775-339a-48b6-88fc-1d6b4252ec3c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.776318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78353775-339a-48b6-88fc-1d6b4252ec3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.798550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-78353775-339a-48b6-88fc-1d6b4252ec3c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:09.799778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-78353775-339a-48b6-88fc-1d6b4252ec3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:09.805193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:09.806338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.806820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.807322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.809437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:09.811264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:09.813648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 144/298] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:09.822424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:09.840712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:09.872954 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 155/299] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:09.874332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c532a-5dd1-432c-be21-bc0625f1b6c7 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.874626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c532a-5dd1-432c-be21-bc0625f1b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.874905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722c532a-5dd1-432c-be21-bc0625f1b6c7 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.876227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 156/300] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:09.882623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f4b93-07d4-4a8c-b66d-3c676a5f58f3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.882909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f4b93-07d4-4a8c-b66d-3c676a5f58f3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.883185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f4b93-07d4-4a8c-b66d-3c676a5f58f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.901992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-cc2f4b93-07d4-4a8c-b66d-3c676a5f58f3 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:09.903168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cc2f4b93-07d4-4a8c-b66d-3c676a5f58f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:09.907020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:09.946464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd930d5f-4574-4505-8120-3ab675510ec8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:09.972589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 157/301] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:09.973257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f4b93-07d4-4a8c-b66d-3c676a5f58f3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:09.994888 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:09.996529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:09.996992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:09.997987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] PATH_INFO: `/v3/groups` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:09.999188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:10.001505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:10.077432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f4b93-07d4-4a8c-b66d-3c676a5f58f3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.111233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.111991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.112272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.112544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.120727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:10.123614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:10.125101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 145/302] 10.222.0.42 () {60 vars in 1134 bytes} [Thu May 26 21:07:09 2022] POST /identity/v3/groups => generated 229 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:07:10.126075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:10.131608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:10.132813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:10.133634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 158/303] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:10.140846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:10.221144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:10.263940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68734a-be8d-4dbf-abc2-eb61d49fc6c4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.267993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f2d3c-6a8c-4fb6-a4ee-280961a24809 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.305531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.306983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.307449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.307863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.308936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.309826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.310280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.310776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.311827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:10.312348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:10.313938 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:10.320962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:10.324736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 159/304] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:10.330122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:10.330985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:10.331833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 146/305] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:10.332538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:10.338754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:10.412520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:10.462295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92da2b-6257-48b0-9923-92c5618ae8d4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.464304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c2a139-8913-4f76-ba44-c3d39f3140d7 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.499361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:10.506091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.511693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.511693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.511693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.511693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:10.511693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:10.512081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.512172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:10.512244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.512812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.513263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] PATH_INFO: `/v3/users/49db91247e9b4dd2a3e7edaf75983f75` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.526419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 147/306] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:10.527814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:10.530032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] RBAC: Authorizing `identity:get_user(user_id=49db91247e9b4dd2a3e7edaf75983f75)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:10.533641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:10.535900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 160/307] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:10 2022] GET /identity/v3/users/49db91247e9b4dd2a3e7edaf75983f75 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:10.541555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:10.650560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6fd7e-339a-492b-ad30-adf62d64c6a2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.684894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ba848-96d0-4c5b-bb09-1457ac7a3b91 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.695995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.696900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.697179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.697453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] PATH_INFO: `/v3/users/7ade02842f6d411ba7215a9f1a76b950` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.699320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] RBAC: Authorizing `identity:get_user(user_id=7ade02842f6d411ba7215a9f1a76b950)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:10.700937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:10.708033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 148/308] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:10 2022] GET /identity/v3/users/7ade02842f6d411ba7215a9f1a76b950 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:10.708702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:10.726172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:10.745313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42665921-c9e9-4da2-8796-96773f3267d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.747672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42665921-c9e9-4da2-8796-96773f3267d2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.748206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42665921-c9e9-4da2-8796-96773f3267d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.748659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42665921-c9e9-4da2-8796-96773f3267d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.749674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42665921-c9e9-4da2-8796-96773f3267d2 None admin] RBAC: Adding query filter params (role.id=c8e1e9ae479b42ddb805138503822330, user.id=49db91247e9b4dd2a3e7edaf75983f75, scope.system=all) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:10.750166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42665921-c9e9-4da2-8796-96773f3267d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:10.752062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42665921-c9e9-4da2-8796-96773f3267d2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:10.768255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 161/309] 10.222.0.42 () {56 vars in 1308 bytes} [Thu May 26 21:07:10 2022] GET /identity/v3/role_assignments?user.id=49db91247e9b4dd2a3e7edaf75983f75&scope.system=all&role.id=c8e1e9ae479b42ddb805138503822330 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:10.785136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:10.829760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d4949-eac0-47a6-8b70-bb64af0e3b9b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:10.869180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:10.870122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.870409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.871223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] PATH_INFO: `/v3/system/users/7ade02842f6d411ba7215a9f1a76b950/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.872849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7ade02842f6d411ba7215a9f1a76b950, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:10.874257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:10.877575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e5eee-bbc7-469d-bdde-d0caee279992 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.877873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e5eee-bbc7-469d-bdde-d0caee279992 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.878135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e5eee-bbc7-469d-bdde-d0caee279992 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.879775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 162/310] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:10.885514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed069-0251-499d-b424-a075507c8dbf None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:10.896510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed069-0251-499d-b424-a075507c8dbf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:10.896510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ed069-0251-499d-b424-a075507c8dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:10.905020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a23ed069-0251-499d-b424-a075507c8dbf None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:10.906153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a23ed069-0251-499d-b424-a075507c8dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:10.983922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 163/311] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:11.006086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ed069-0251-499d-b424-a075507c8dbf None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:11.021016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 149/312] 10.222.0.42 () {58 vars in 1334 bytes} [Thu May 26 21:07:10 2022] PUT /identity/v3/system/users/7ade02842f6d411ba7215a9f1a76b950/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:11.127622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ed069-0251-499d-b424-a075507c8dbf None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:11.144673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:11.167967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:11.169125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:11.169629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:11.170124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:11.182495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:11.191030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:11.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:11.192868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:11.195782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 164/313] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:11.205810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:11.314467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ea767e-ae1c-4ed5-bba3-fca0475d65ce None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:11.342591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:11.353935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:11.357523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:11.357523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:11.357523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:11.357523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:11.357523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:11.358163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:11.368660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 150/314] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:11.376871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:11.482610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882edd41-65b8-4f03-b724-1a8768a46792 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:11.517554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:11.518582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:11.518957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:11.519323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] PATH_INFO: `/v3/users/0a08c5eda47f41d8a1d635e60b73216f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:11.520674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] RBAC: Authorizing `identity:get_user(user_id=0a08c5eda47f41d8a1d635e60b73216f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:11.522369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:11.524482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 165/315] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:11 2022] GET /identity/v3/users/0a08c5eda47f41d8a1d635e60b73216f => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:11.529811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:11.634975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf816d20-1eae-480b-b96c-02b8e096d48c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:11.670225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:11.670882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:11.671166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:11.671445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:11.747111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:11.753794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:11.755287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:11.858233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 151/316] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:11.863787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:11.912838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:11.977162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308134af-c88b-41d9-b7e4-d9cafd859c50 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:12.014813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e29812-1976-48fd-9aeb-0de7341d3d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:12.016477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e29812-1976-48fd-9aeb-0de7341d3d82 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:12.017005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e29812-1976-48fd-9aeb-0de7341d3d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:12.017481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e29812-1976-48fd-9aeb-0de7341d3d82 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:12.019025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e29812-1976-48fd-9aeb-0de7341d3d82 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:12.020721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e29812-1976-48fd-9aeb-0de7341d3d82 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:12.022935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 166/317] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:11 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:12.029015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:12.117194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:12.132581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeecc84-cbd2-43da-b0c1-71e18b49bc84 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:12.168245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd23f5-8bfe-499c-a586-a4b3bf624e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:12.168911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd23f5-8bfe-499c-a586-a4b3bf624e45 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:12.169195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd23f5-8bfe-499c-a586-a4b3bf624e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:12.169475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd23f5-8bfe-499c-a586-a4b3bf624e45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:12.170307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd23f5-8bfe-499c-a586-a4b3bf624e45 None admin] RBAC: Adding query filter params (role.id=c8e1e9ae479b42ddb805138503822330, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9, user.id=0a08c5eda47f41d8a1d635e60b73216f) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:12.170620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd23f5-8bfe-499c-a586-a4b3bf624e45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:12.172066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd23f5-8bfe-499c-a586-a4b3bf624e45 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:12.178372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 152/318] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:12 2022] GET /identity/v3/role_assignments?user.id=0a08c5eda47f41d8a1d635e60b73216f&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=c8e1e9ae479b42ddb805138503822330 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:12.506392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a887c-1156-46f7-b99d-102ca89e5270 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:12.506768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a887c-1156-46f7-b99d-102ca89e5270 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:12.506964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a887c-1156-46f7-b99d-102ca89e5270 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:12.508480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 167/319] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:12.513639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f47ab6-f7a7-48c9-aada-70985cd5c6a8 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:12.513999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f47ab6-f7a7-48c9-aada-70985cd5c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:12.514347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f47ab6-f7a7-48c9-aada-70985cd5c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:12.531291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f6f47ab6-f7a7-48c9-aada-70985cd5c6a8 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:12.532585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f6f47ab6-f7a7-48c9-aada-70985cd5c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:12.603054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 153/320] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:12.625136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a887c-1156-46f7-b99d-102ca89e5270 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:12.737924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a887c-1156-46f7-b99d-102ca89e5270 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:12.749391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:12.773276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e5a237-7285-4a84-bf19-f4468bc773ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:12.773993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5a237-7285-4a84-bf19-f4468bc773ce None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:12.774273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5a237-7285-4a84-bf19-f4468bc773ce None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:12.774542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5a237-7285-4a84-bf19-f4468bc773ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:12.775710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5a237-7285-4a84-bf19-f4468bc773ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:12.777786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e5a237-7285-4a84-bf19-f4468bc773ce None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:12.779945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 168/321] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:12.785438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f47ab6-f7a7-48c9-aada-70985cd5c6a8 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:12.867031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:12.887796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f47ab6-f7a7-48c9-aada-70985cd5c6a8 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:12.923635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c7875-4e18-4430-8a41-d168dfe16840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:12.924360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7875-4e18-4430-8a41-d168dfe16840 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:12.924644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7875-4e18-4430-8a41-d168dfe16840 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:12.924922 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c7875-4e18-4430-8a41-d168dfe16840 None admin] PATH_INFO: `/v3/groups` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:12.925652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c7875-4e18-4430-8a41-d168dfe16840 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:12.927163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4c7875-4e18-4430-8a41-d168dfe16840 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:13.055850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 154/322] 10.222.0.42 () {60 vars in 1134 bytes} [Thu May 26 21:07:12 2022] POST /identity/v3/groups => generated 226 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:13.116859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c1b94-0828-4c09-97f8-103c61b18a45 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.117406 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c1b94-0828-4c09-97f8-103c61b18a45 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.117406 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c1b94-0828-4c09-97f8-103c61b18a45 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.119024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 169/323] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:13.124837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764ceb-ebed-4138-9188-ade38dbce591 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.125185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764ceb-ebed-4138-9188-ade38dbce591 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.125411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5764ceb-ebed-4138-9188-ade38dbce591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.145165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-a5764ceb-ebed-4138-9188-ade38dbce591 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:13.146439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a5764ceb-ebed-4138-9188-ade38dbce591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:13.228165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 155/324] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:13.251401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c1b94-0828-4c09-97f8-103c61b18a45 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:13.332819 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75313416-b9d7-490c-9469-d74177d683c1 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.333038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75313416-b9d7-490c-9469-d74177d683c1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.333382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75313416-b9d7-490c-9469-d74177d683c1 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.334763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 156/325] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:13.340750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29397-4fd1-4fc9-8169-a06fffeea923 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.341322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29397-4fd1-4fc9-8169-a06fffeea923 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.341779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf29397-4fd1-4fc9-8169-a06fffeea923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.358863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3bf29397-4fd1-4fc9-8169-a06fffeea923 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:13.360241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3bf29397-4fd1-4fc9-8169-a06fffeea923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:13.374689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c1b94-0828-4c09-97f8-103c61b18a45 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:13.423260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:13.424134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.424382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.425157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.440142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:13.451798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:13.452718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:13.453507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:13.456627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 170/326] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:13.462294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:13.463221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:13.469092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 157/327] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:13.491007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf29397-4fd1-4fc9-8169-a06fffeea923 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:13.518415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:13.601622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37730dcd-3b9b-4a6b-98cc-0c6554b2ef72 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:13.619148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf29397-4fd1-4fc9-8169-a06fffeea923 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:13.653237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:13.653924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.654210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.654485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.655714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:13.656069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:13.658057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:13.665259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:13.666694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.667186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.668251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.669122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 171/328] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:13.680794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:13.687880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:13.697799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:13.699294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:13.700166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 158/329] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:13.703764 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:13.775258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:13.827425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a0491-e2ef-49f9-abfb-cbcdf9d2314c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:13.834176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f00a63-1bd1-4504-b369-8a4c8a097de1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:13.836088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:13.868017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:13.868017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.868017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.868017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.868731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:13.868731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:13.868731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:13.878173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 159/330] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:13.880483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:13.881374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:13.881659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:13.882205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] PATH_INFO: `/v3/users/49db91247e9b4dd2a3e7edaf75983f75` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:13.883309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:13.883432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] RBAC: Authorizing `identity:get_user(user_id=49db91247e9b4dd2a3e7edaf75983f75)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:13.884930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:13.887801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 172/331] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:13 2022] GET /identity/v3/users/49db91247e9b4dd2a3e7edaf75983f75 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:13.893386 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:14.015666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eff4df-a4dd-4631-9f95-acba003d8ae7 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:14.018130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7861c-33d6-436c-b541-185b962c1ce7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:14.055526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:14.076833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:14.077772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:14.078058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:14.078329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] PATH_INFO: `/v3/users/7ade02842f6d411ba7215a9f1a76b950` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:14.079659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0279de0-0a76-466e-b635-bea5747c5566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:14.080557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0279de0-0a76-466e-b635-bea5747c5566 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:14.080840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0279de0-0a76-466e-b635-bea5747c5566 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:14.081116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0279de0-0a76-466e-b635-bea5747c5566 None admin] PATH_INFO: `/v3/system/users/49db91247e9b4dd2a3e7edaf75983f75/roles/c8e1e9ae479b42ddb805138503822330` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:14.082650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] RBAC: Authorizing `identity:get_user(user_id=7ade02842f6d411ba7215a9f1a76b950)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:14.084164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:14.086016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 160/332] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:13 2022] GET /identity/v3/users/7ade02842f6d411ba7215a9f1a76b950 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:14.086593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0279de0-0a76-466e-b635-bea5747c5566 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=49db91247e9b4dd2a3e7edaf75983f75, role_id=c8e1e9ae479b42ddb805138503822330)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:14.088002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0279de0-0a76-466e-b635-bea5747c5566 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:14.099144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:14.145802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 173/333] 10.222.0.42 () {58 vars in 1334 bytes} [Thu May 26 21:07:13 2022] PUT /identity/v3/system/users/49db91247e9b4dd2a3e7edaf75983f75/roles/c8e1e9ae479b42ddb805138503822330 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:14.241442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c88e1b8-00de-444e-8c29-b0e6f8ba6477 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:14.290209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35266967-feb9-4c3a-b220-7f786dc58f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:14.290918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266967-feb9-4c3a-b220-7f786dc58f53 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:14.291208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266967-feb9-4c3a-b220-7f786dc58f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:14.291488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35266967-feb9-4c3a-b220-7f786dc58f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:14.292894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266967-feb9-4c3a-b220-7f786dc58f53 None admin] RBAC: Adding query filter params (role.id=1183a502796d46e580a92e3c20f04103, user.id=7ade02842f6d411ba7215a9f1a76b950, scope.system=all) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:14.293213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266967-feb9-4c3a-b220-7f786dc58f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:14.294669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35266967-feb9-4c3a-b220-7f786dc58f53 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:14.310589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 161/334] 10.222.0.42 () {56 vars in 1308 bytes} [Thu May 26 21:07:14 2022] GET /identity/v3/role_assignments?user.id=7ade02842f6d411ba7215a9f1a76b950&scope.system=all&role.id=1183a502796d46e580a92e3c20f04103 => generated 510 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:16.413667 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e887b-e233-4d3c-b166-99d404c605bb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.414693 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e887b-e233-4d3c-b166-99d404c605bb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.415204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e887b-e233-4d3c-b166-99d404c605bb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.417062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 174/335] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:16.425376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d026d2-9e40-4829-bcff-765af551ea22 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.426283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d026d2-9e40-4829-bcff-765af551ea22 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.426768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d026d2-9e40-4829-bcff-765af551ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.438318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6037a9-6b9b-460d-ba03-6267b672724e None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.438923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6037a9-6b9b-460d-ba03-6267b672724e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.439415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6037a9-6b9b-460d-ba03-6267b672724e None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.441023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 175/336] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:16.446635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3f4c0-9f1b-42bb-ab16-8464ce11a220 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.447160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3f4c0-9f1b-42bb-ab16-8464ce11a220 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.447732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3f4c0-9f1b-42bb-ab16-8464ce11a220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.474934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-61d026d2-9e40-4829-bcff-765af551ea22 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:16.476589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-61d026d2-9e40-4829-bcff-765af551ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:16.478358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-80f3f4c0-9f1b-42bb-ab16-8464ce11a220 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:16.482520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-80f3f4c0-9f1b-42bb-ab16-8464ce11a220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:16.552117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:16.585356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:16.627056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 176/337] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:16.645718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 162/338] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:16.679887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3f4c0-9f1b-42bb-ab16-8464ce11a220 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:16.706732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d026d2-9e40-4829-bcff-765af551ea22 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:16.830946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3f4c0-9f1b-42bb-ab16-8464ce11a220 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:16.862217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d026d2-9e40-4829-bcff-765af551ea22 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:16.886080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ed465-47cd-4362-bf5a-f4faaaefb055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:16.888439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ed465-47cd-4362-bf5a-f4faaaefb055 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.888719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ed465-47cd-4362-bf5a-f4faaaefb055 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.888987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ed465-47cd-4362-bf5a-f4faaaefb055 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.906921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:16.907730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.908039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.908316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.913880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ed465-47cd-4362-bf5a-f4faaaefb055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-c83ed465-47cd-4362-bf5a-f4faaaefb055 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:16.923204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:16.927476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 177/339] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:16.928450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61d049-53d5-40d5-a2e0-36337d5aa8d3 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.929162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61d049-53d5-40d5-a2e0-36337d5aa8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.929162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf61d049-53d5-40d5-a2e0-36337d5aa8d3 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.929162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 178/340] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:16.934760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97de5a-b9f3-4ece-a357-d7954fe2d964 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:16.934972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97de5a-b9f3-4ece-a357-d7954fe2d964 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:16.935412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b97de5a-b9f3-4ece-a357-d7954fe2d964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:16.944739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:16.953906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:16.954991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:16.955843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:16.964776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 163/341] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:16.965993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:16.970277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7b97de5a-b9f3-4ece-a357-d7954fe2d964 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:16.971638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7b97de5a-b9f3-4ece-a357-d7954fe2d964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:17.007948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:17.048022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:17.093331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 179/342] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:17.094620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97de5a-b9f3-4ece-a357-d7954fe2d964 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:17.127753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba32a83-c2cb-41c2-a243-2f1cf328f58e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:17.187486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:17.188419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:17.188869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:17.189342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:17.190343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:17.190794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:17.192353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:17.218119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 164/343] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:17.218996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:17.260464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b97de5a-b9f3-4ece-a357-d7954fe2d964 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:17.336484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:17.337416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:17.337875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:17.338295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:17.338729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:17.339401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] RBAC: Adding query filter params (name=member) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:17.339880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:17.341491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:17.350665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438b49e-0b0d-4662-aef8-ac13b0e26ea2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:17.363444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 180/344] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:17.364555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:17.397280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:17.425212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:17.426295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:17.428199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:17.428441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:17.460385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:17.479659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:17.481256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:17.482063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 165/345] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:17.482775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:17.579264 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0ce6a-42fd-4e46-bdfd-b11aa35e121e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:17.606813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cec0d-395a-4b2b-8f9f-682e74543bd0 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:17.629028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:17.630030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:17.632153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:17.632434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:17.650221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87167f91-4626-49c0-9a48-4dbdb94d766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:17.651344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87167f91-4626-49c0-9a48-4dbdb94d766a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:17.652356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87167f91-4626-49c0-9a48-4dbdb94d766a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:17.653205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87167f91-4626-49c0-9a48-4dbdb94d766a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:17.709265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:17.728153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:17.730202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:17.750657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:17.765322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87167f91-4626-49c0-9a48-4dbdb94d766a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:17.766981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87167f91-4626-49c0-9a48-4dbdb94d766a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:17.850602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 181/346] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:17.851609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:17.893648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 166/347] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:17.895311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b564508-e39b-4f77-8a1d-48c2619db402 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:17.895861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b564508-e39b-4f77-8a1d-48c2619db402 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:17.896344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b564508-e39b-4f77-8a1d-48c2619db402 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:17.898099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 167/348] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:17.899414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cddf3a-00bc-49b0-9d3e-d77940a55017 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:17.899703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cddf3a-00bc-49b0-9d3e-d77940a55017 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:17.903970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cddf3a-00bc-49b0-9d3e-d77940a55017 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:17.905433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 168/349] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:17.906032 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cddf3a-00bc-49b0-9d3e-d77940a55017 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:17.980221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa8696-e6de-4a4f-8e27-7f5aa5c9fb4d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:17.980854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:17.996726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:18.024267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.025249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.025523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.026277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:18.026581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:18.029567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:18.038580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cddf3a-00bc-49b0-9d3e-d77940a55017 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.038881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 182/350] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:18.042184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.081545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16efbd-639d-424b-b32b-3156cc450ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.082417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16efbd-639d-424b-b32b-3156cc450ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.082699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16efbd-639d-424b-b32b-3156cc450ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.082974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16efbd-639d-424b-b32b-3156cc450ac1 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.084200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16efbd-639d-424b-b32b-3156cc450ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:18.085718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16efbd-639d-424b-b32b-3156cc450ac1 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:18.087600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 169/351] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:17 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:18.088675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0effd-5f63-4ff6-96ae-5b46e46f08f7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.088946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0effd-5f63-4ff6-96ae-5b46e46f08f7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.089757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0effd-5f63-4ff6-96ae-5b46e46f08f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.113135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-8cb0effd-5f63-4ff6-96ae-5b46e46f08f7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:18.114330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8cb0effd-5f63-4ff6-96ae-5b46e46f08f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:18.157957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008f3c3-76b9-452c-ad63-f31c1d766d43 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.197491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 170/352] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:18.201245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fc016-20aa-4d4e-b28e-9350a43d9405 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.201245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fc016-20aa-4d4e-b28e-9350a43d9405 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.201245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fc016-20aa-4d4e-b28e-9350a43d9405 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.201508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.201596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 171/353] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:18.202025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.202235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c449e-91ec-4fd1-a046-33a4969ea3c0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.202330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.202526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c449e-91ec-4fd1-a046-33a4969ea3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.202655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.202833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198c449e-91ec-4fd1-a046-33a4969ea3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.204691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:18.206361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:18.208641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 183/354] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:18.209631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.220995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-198c449e-91ec-4fd1-a046-33a4969ea3c0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:18.222189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-198c449e-91ec-4fd1-a046-33a4969ea3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:18.264145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:18.291150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:18.308994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 172/355] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:18.309859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c449e-91ec-4fd1-a046-33a4969ea3c0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.332124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be52adb-759e-44d8-8766-e76e9079dc82 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.371083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.371757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.372076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.372358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.426437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198c449e-91ec-4fd1-a046-33a4969ea3c0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.454425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:18.455943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:18.466999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f104348-7fec-4482-aa8d-cffa4af8f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.468259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104348-7fec-4482-aa8d-cffa4af8f4cd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.468548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104348-7fec-4482-aa8d-cffa4af8f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.468827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f104348-7fec-4482-aa8d-cffa4af8f4cd None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.477100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f104348-7fec-4482-aa8d-cffa4af8f4cd None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:18.478568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f104348-7fec-4482-aa8d-cffa4af8f4cd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:18.481042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 173/356] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:18.482481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca70c4-d442-4d6a-9336-869fdd6b85c7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.483042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca70c4-d442-4d6a-9336-869fdd6b85c7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.483474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca70c4-d442-4d6a-9336-869fdd6b85c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.503020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f8ca70c4-d442-4d6a-9336-869fdd6b85c7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:18.504452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f8ca70c4-d442-4d6a-9336-869fdd6b85c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:18.511803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:18.567553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:18.583365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 184/357] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:18.583365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.598085 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 174/358] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:18.599251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca70c4-d442-4d6a-9336-869fdd6b85c7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.720250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22bc07-9d7d-4fe5-a947-80744b99c432 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.741150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca70c4-d442-4d6a-9336-869fdd6b85c7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.767608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.768526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.768811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.769088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.770978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:18.772495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:18.774317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 185/359] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:18.774939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.779818 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.780641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.780920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.781191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.794602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:18.798326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:18.807179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:18.808866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:18.809651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 175/360] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:18.810289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.860206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:18.903642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebedb928-d05d-4b08-80b1-84afb92847e8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.928677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499db96-5fdb-4ac5-902a-aba9130c8db8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:18.946409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.947613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.948434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.949501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.961914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:18.968222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:18.968876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:18.969159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:18.969434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:18.970298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=c23de713167345639a2085add0a54ae6, scope.project.id=a2cdddc85fc3414a8028b384e27eabad) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:18.970468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:18.971309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:18.972246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:18.972849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:18.972915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 186/361] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:18.972977 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:18.973040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:18.978810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 176/362] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=c23de713167345639a2085add0a54ae6 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:18.980542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.084284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.128935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db476f9-6656-4696-832a-7d45d8136463 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.142617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d919843-1276-475d-8a9c-769d460e172e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.152209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.168127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.169006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.169284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.169556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.170791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.172393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:19.174375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 187/363] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:19.175036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.187994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.189462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.190399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.190968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.202664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:19.210627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:19.211672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:19.212615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 177/364] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:19.213382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.295202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e5600-d80a-4954-9f74-c62ad1dea51d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.318912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea8494-50a5-4b50-a2e0-24c5a0401faf None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.330436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.331097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.331383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.331658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.332543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=a2cdddc85fc3414a8028b384e27eabad) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:19.332910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.334373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:19.341078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 188/365] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=1183a502796d46e580a92e3c20f04103 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:19.341794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.346862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.366949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.368558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.369306 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.369756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.370953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:19.371437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.373947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:19.386589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 178/366] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:19.391769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.425416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.488143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24d339c-7de3-438e-baf1-5f60d6dedee9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.516262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8004a6-0130-4d87-bd98-e9327c6e5535 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.527081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.528301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.528301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.528657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.529510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:19.529918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.531395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:19.546907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 189/367] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:19.548316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.566555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.567775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.567972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.568350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.569814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.571454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:19.573253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 179/368] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:19.579378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.645204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.682379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f0c16f-3190-4728-8467-fe11a4ddefb4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.684925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.691508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b351633-7761-41c5-a813-5cfb658bec57 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.722152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.723069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.723550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.724121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.725170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:19.725663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.727389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:19.730618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.731361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.731562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.731853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.738703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 190/369] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:19.739891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.803418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.804915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:19.852138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea10d1c-d292-41ae-bd44-83ef7edb9ae9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:19.881818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.891019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:19.892003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:19.892631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:19.893138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:19.909116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:19.912683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 180/370] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:19.913906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:19.970836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:19.973087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.041831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e269d4f7-e822-4aee-aeac-cec59972fa9d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:20.079397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:20.080217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:20.080503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:20.080781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:20.081621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:20.081940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:20.083701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.090291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 181/371] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=1183a502796d46e580a92e3c20f04103 => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:20.091124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.092752 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 191/372] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:20.093707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.127462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:20.165709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:20.213908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52b0e6-e2bb-49d3-ba7a-d78c1aa559b3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:20.240728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea881128-4ab0-4524-adc5-972ac8fc631a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:20.252796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:20.253799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:20.254089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:20.254365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] PATH_INFO: `/v3/users/49db91247e9b4dd2a3e7edaf75983f75` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:20.255861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] RBAC: Authorizing `identity:get_user(user_id=49db91247e9b4dd2a3e7edaf75983f75)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:20.257348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.259368 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 182/373] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/users/49db91247e9b4dd2a3e7edaf75983f75 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:20.260262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.305589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:20.306477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:20.306760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:20.307034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:20.310412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:20.311914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.318788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 192/374] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:20.319686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.431997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8414090-1fd3-40a3-ab10-f8b16355d033 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:20.444937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:20.454504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072eec8f-09df-4729-b4a8-27a640f6024d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:20.476480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:20.491425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:20.492323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:20.492609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:20.492883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] PATH_INFO: `/v3/projects/213a059b002f40c1bddeb6d9f1da1459` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:20.494971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] RBAC: Authorizing `identity:get_project(project_id=213a059b002f40c1bddeb6d9f1da1459)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:20.497027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.498735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 183/375] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/projects/213a059b002f40c1bddeb6d9f1da1459 => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:20.499509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.521570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:20.522302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:20.522586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:20.522857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:20.523826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] RBAC: Adding query filter params (role.id=c8e1e9ae479b42ddb805138503822330, user.id=49db91247e9b4dd2a3e7edaf75983f75, scope.system=all) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:20.524139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:20.528242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.556809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 193/376] 10.222.0.42 () {56 vars in 1308 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/role_assignments?user.id=49db91247e9b4dd2a3e7edaf75983f75&scope.system=all&role.id=c8e1e9ae479b42ddb805138503822330 => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:20.557678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.721296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea855fd-196f-4d43-9620-23aeb4297677 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:20.792049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8599ab1-41d2-4152-908b-c1d387190d69 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:20.824768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:20.825675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:20.825960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:20.826240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:20.827561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:20.829073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.831769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 184/377] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:20.836407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.862644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:20.863509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:20.863921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:20.864202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] PATH_INFO: `/v3/groups/7d93d0b9ae4844a8b8b3157e74ab0372` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:20.871623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] RBAC: Authorizing `identity:get_group(group_id=7d93d0b9ae4844a8b8b3157e74ab0372)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:20.873218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:20.875093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 194/378] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/groups/7d93d0b9ae4844a8b8b3157e74ab0372 => generated 229 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:20.880634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:20.911491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:20.944933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:21.000603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524451d-4251-41a2-9fb6-8b0adcde5bcd None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:21.051241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0a05-8293-454f-86f1-140d6bddd327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:21.051935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0a05-8293-454f-86f1-140d6bddd327 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:21.052218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0a05-8293-454f-86f1-140d6bddd327 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:21.052490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0a05-8293-454f-86f1-140d6bddd327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:21.053323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf0a05-8293-454f-86f1-140d6bddd327 None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=c23de713167345639a2085add0a54ae6, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:21.053634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf0a05-8293-454f-86f1-140d6bddd327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:21.055073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf0a05-8293-454f-86f1-140d6bddd327 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:21.071861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 185/379] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=c23de713167345639a2085add0a54ae6 => generated 260 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:21.079497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb43f0-8aaf-4559-8dd2-b73f7025cf95 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:21.126045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7e36e-408c-464f-9c9c-b435adff9c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:21.126798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e36e-408c-464f-9c9c-b435adff9c05 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:21.127070 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e36e-408c-464f-9c9c-b435adff9c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:21.127344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7e36e-408c-464f-9c9c-b435adff9c05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:21.128217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7e36e-408c-464f-9c9c-b435adff9c05 None admin] RBAC: Adding query filter params (group.id=7d93d0b9ae4844a8b8b3157e74ab0372, role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=213a059b002f40c1bddeb6d9f1da1459) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:21.128531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7e36e-408c-464f-9c9c-b435adff9c05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:21.130015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7e36e-408c-464f-9c9c-b435adff9c05 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:21.137420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 195/380] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:20 2022] GET /identity/v3/role_assignments?group.id=7d93d0b9ae4844a8b8b3157e74ab0372&scope.project.id=213a059b002f40c1bddeb6d9f1da1459&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:21.525566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c6809-e5e6-4362-b721-2c7074d53dbc None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:21.525939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c6809-e5e6-4362-b721-2c7074d53dbc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:21.526117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7c6809-e5e6-4362-b721-2c7074d53dbc None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:21.528401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 186/381] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:21.533661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297eee96-a677-4acf-8b0a-833dc462b1c7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:21.534228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297eee96-a677-4acf-8b0a-833dc462b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:21.534707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297eee96-a677-4acf-8b0a-833dc462b1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:21.554908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-297eee96-a677-4acf-8b0a-833dc462b1c7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:21.556497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-297eee96-a677-4acf-8b0a-833dc462b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:21.634304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 196/382] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:21.656836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c6809-e5e6-4362-b721-2c7074d53dbc None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:21.779836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7c6809-e5e6-4362-b721-2c7074d53dbc None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:21.807089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:21.815542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e267a-d303-478c-8390-5e790ae5c40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:21.816461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e267a-d303-478c-8390-5e790ae5c40e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:21.816750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e267a-d303-478c-8390-5e790ae5c40e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:21.817030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e267a-d303-478c-8390-5e790ae5c40e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:21.829429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e267a-d303-478c-8390-5e790ae5c40e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-bc2e267a-d303-478c-8390-5e790ae5c40e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:21.837739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:21.838637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:21.840037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 187/383] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:21.845445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297eee96-a677-4acf-8b0a-833dc462b1c7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:21.860489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:21.946152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926f8e5-925d-41fa-af6e-f26539483106 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:21.946477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926f8e5-925d-41fa-af6e-f26539483106 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:21.946725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926f8e5-925d-41fa-af6e-f26539483106 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:21.948281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 188/384] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:21.954970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf969bec-2932-46ef-93ab-7f35e8c71511 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:21.955313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf969bec-2932-46ef-93ab-7f35e8c71511 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:21.955542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf969bec-2932-46ef-93ab-7f35e8c71511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:21.972087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297eee96-a677-4acf-8b0a-833dc462b1c7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:21.988808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bf969bec-2932-46ef-93ab-7f35e8c71511 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:21.990035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bf969bec-2932-46ef-93ab-7f35e8c71511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:22.019045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:22.020653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:22.021310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:22.021912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:22.023122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:22.023837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:22.025452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:22.038519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 197/385] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:22.044359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:22.138795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 189/386] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:22.158387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:22.179234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf969bec-2932-46ef-93ab-7f35e8c71511 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:22.185958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3799b3-9771-40b4-9ee7-ff2eff5bdcbd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:22.224609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:22.225887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:22.226376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:22.226875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:22.228445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:22.230459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:22.233234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 198/387] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:22 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:22.237826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:22.327727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf969bec-2932-46ef-93ab-7f35e8c71511 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:22.348557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:22.367386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:22.368610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:22.369103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:22.369703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:22.382483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:22.391598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:22.392519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:22.393380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:22.396317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 190/388] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:22.402276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:22.489951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a963b81-fc15-4633-a6dd-481c07961c1d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:22.513950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d47255-4f4e-45f3-b7bd-373120fd4f5c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:22.551520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:22.552748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:22.553253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:22.553731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:22.554681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:22.555281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:22.557005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:22.567341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 191/389] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:22.574367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:22.576469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:22.577369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:22.577659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:22.577992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:22.579304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:22.581015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:22.587573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 199/390] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:22 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:22.588574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:22.619100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:22.659386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:22.768196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edc918-3ecf-4c1e-8414-7fd52b116c9b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:22.813040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28828d4-5c05-4de6-bb0b-35ee66813214 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:22.837513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:22.838365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:22.838651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:22.838942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:22.841538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:22.843007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:22.847231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 192/391] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:22 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:22.851068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:22.879135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:22.880062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:22.880345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:22.880621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad/groups/f8812941ebfb4a128bbb6888b1765cee/roles/c23de713167345639a2085add0a54ae6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:22.885023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cdddc85fc3414a8028b384e27eabad, group_id=f8812941ebfb4a128bbb6888b1765cee, role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:22.887016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:22.974394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 200/392] 10.222.0.42 () {58 vars in 1442 bytes} [Thu May 26 21:07:22 2022] PUT /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad/groups/f8812941ebfb4a128bbb6888b1765cee/roles/c23de713167345639a2085add0a54ae6 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:07:22.975701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b048-da2f-4b84-a04a-fa6975b20eb4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:22.998333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:23.054595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3177c-f518-473b-a623-3658efb52572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:23.055487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3177c-f518-473b-a623-3658efb52572 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:23.055785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3177c-f518-473b-a623-3658efb52572 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:23.056068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3177c-f518-473b-a623-3658efb52572 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:23.060660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3177c-f518-473b-a623-3658efb52572 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:23.062054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3177c-f518-473b-a623-3658efb52572 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:23.063838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 193/393] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:22 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:23.069967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:23.172868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:23.174118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a319fa-15fe-462c-a25c-8ef476256a14 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:23.174409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a319fa-15fe-462c-a25c-8ef476256a14 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:23.174690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a319fa-15fe-462c-a25c-8ef476256a14 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:23.176087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 194/394] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:23.186158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6c4b7-07b9-4313-8b59-8e074c416387 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:23.188373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6c4b7-07b9-4313-8b59-8e074c416387 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:23.188373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6c4b7-07b9-4313-8b59-8e074c416387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:23.211150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290f179-92dc-4698-9f9b-88dcb6380293 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:23.220791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2bf6c4b7-07b9-4313-8b59-8e074c416387 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:23.221996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2bf6c4b7-07b9-4313-8b59-8e074c416387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:23.259785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:23.260691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:23.261601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:23.261881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad/groups/f8812941ebfb4a128bbb6888b1765cee/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:23.269506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] RBAC: Authorizing `identity:create_grant(project_id=a2cdddc85fc3414a8028b384e27eabad, group_id=f8812941ebfb4a128bbb6888b1765cee, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:23.270975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:23.315012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 195/395] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:23.338183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6c4b7-07b9-4313-8b59-8e074c416387 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:23.340391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 201/396] 10.222.0.42 () {58 vars in 1442 bytes} [Thu May 26 21:07:23 2022] PUT /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad/groups/f8812941ebfb4a128bbb6888b1765cee/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:23.494688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6c4b7-07b9-4313-8b59-8e074c416387 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:23.506712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:23.543829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:23.544668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:23.545035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:23.545640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:23.564374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:23.572713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:23.573693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:23.574552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 196/397] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:23.579980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:23.750670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e554d-0767-403c-bf71-2b673a630157 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:23.802951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:23.809579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c33ef-a0f2-464b-9cd2-fb02366c106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:23.810260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c33ef-a0f2-464b-9cd2-fb02366c106b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:23.810546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c33ef-a0f2-464b-9cd2-fb02366c106b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:23.810821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c33ef-a0f2-464b-9cd2-fb02366c106b None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:23.811572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c33ef-a0f2-464b-9cd2-fb02366c106b None admin] RBAC: Adding query filter params (name=member) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:23.811909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c33ef-a0f2-464b-9cd2-fb02366c106b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:23.813371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c33ef-a0f2-464b-9cd2-fb02366c106b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:23.829028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 202/398] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:23.834775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:23.977776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fb451-0e89-4228-9c9c-6219050b3e10 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.004171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1990d9-f9dc-4544-a03b-098b5031e1fc None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.004670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1990d9-f9dc-4544-a03b-098b5031e1fc None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.004670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1990d9-f9dc-4544-a03b-098b5031e1fc None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.005999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 203/399] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:24.013067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1769ad7-c39e-4dce-ab53-165cc0fcef81 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.013322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1769ad7-c39e-4dce-ab53-165cc0fcef81 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.013642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1769ad7-c39e-4dce-ab53-165cc0fcef81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.025333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5f9f87-2a50-494c-8182-5ad9c7cc83be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:24.026201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f9f87-2a50-494c-8182-5ad9c7cc83be None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.027022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f9f87-2a50-494c-8182-5ad9c7cc83be None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.027305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5f9f87-2a50-494c-8182-5ad9c7cc83be None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.028516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f9f87-2a50-494c-8182-5ad9c7cc83be None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:24.030000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5f9f87-2a50-494c-8182-5ad9c7cc83be None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:24.031917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 197/400] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:23 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:24.033111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb20e-f546-4023-ba31-92dcec74df9a None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.033401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb20e-f546-4023-ba31-92dcec74df9a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.033681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb20e-f546-4023-ba31-92dcec74df9a None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.034898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 198/401] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:24.035447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f1769ad7-c39e-4dce-ab53-165cc0fcef81 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:24.040456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb20e-f546-4023-ba31-92dcec74df9a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.041184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f1769ad7-c39e-4dce-ab53-165cc0fcef81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:24.086208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:24.123775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 204/402] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:24.123775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ea4cf-2e21-44ee-9ac7-510a8d7453fd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.123775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ea4cf-2e21-44ee-9ac7-510a8d7453fd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.123775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93ea4cf-2e21-44ee-9ac7-510a8d7453fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.141585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f93ea4cf-2e21-44ee-9ac7-510a8d7453fd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:24.142725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f93ea4cf-2e21-44ee-9ac7-510a8d7453fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:24.177855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb20e-f546-4023-ba31-92dcec74df9a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.223766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 205/403] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:24.223766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ea4cf-2e21-44ee-9ac7-510a8d7453fd None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.224393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:24.225555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.226035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.226506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.228029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:24.230275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:24.232645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 199/404] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:24 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:24.238333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.277750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:24.326195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93ea4cf-2e21-44ee-9ac7-510a8d7453fd None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.353331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48190562-80fb-4d0d-b820-a4205210eef0 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.355723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:24.362880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:24.363684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.364390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.364390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.375627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:24.383617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:24.384544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:24.385402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 206/405] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:07:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:24.386153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.402262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:24.403529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.404064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.404595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/groups/f8812941ebfb4a128bbb6888b1765cee/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.406849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dad1ade88dd4d0b8c70347b618248c9, group_id=f8812941ebfb4a128bbb6888b1765cee, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:24.408531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:24.505120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 200/406] 10.222.0.42 () {58 vars in 1442 bytes} [Thu May 26 21:07:24 2022] PUT /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/groups/f8812941ebfb4a128bbb6888b1765cee/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:07:24.506285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.508242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7e48e-f755-4780-814d-98bd9361428f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.539154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:24.584420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:24.585659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.586145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.586622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.610306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:24.621987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:24.622965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:24.623823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:24.636833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 207/407] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:24.641866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.720946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5eb384-bd04-40a4-86b0-f76a0b52bd63 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.765577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:24.766662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.767193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.767769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.768847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:24.769421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:24.770977 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:24.781291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 201/408] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:24.793682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.831927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:24.836533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304fbaf5-d8eb-419d-aeff-a58648f8c663 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.881067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:24.882145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:24.882663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:24.883312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:24.884870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:24.885392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:24.888057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:24.900627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 208/409] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:24.908800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:24.931448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5b4a6-6200-4e32-acb1-69c4faaa058d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:24.978300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:25.002181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:25.003047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.003329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.003602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] PATH_INFO: `/v3/groups/7d93d0b9ae4844a8b8b3157e74ab0372` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.005979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] RBAC: Authorizing `identity:get_group(group_id=7d93d0b9ae4844a8b8b3157e74ab0372)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:25.007487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:25.012532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 202/410] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:24 2022] GET /identity/v3/groups/7d93d0b9ae4844a8b8b3157e74ab0372 => generated 229 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:25.019666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:25.118593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab744dec-be2a-49a4-a223-7ad54e268669 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:25.171287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:25.172246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.172534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.172812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.174132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:25.174942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:25.176577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:25.178772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 209/411] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:24 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:25.184385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:25.184747 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb4164-8179-4674-8e52-ef29729d6942 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:25.224185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:25.225061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.225349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.225630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] PATH_INFO: `/v3/projects/213a059b002f40c1bddeb6d9f1da1459` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.226850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] RBAC: Authorizing `identity:get_project(project_id=213a059b002f40c1bddeb6d9f1da1459)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:25.228342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:25.230913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 203/412] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:25 2022] GET /identity/v3/projects/213a059b002f40c1bddeb6d9f1da1459 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:25.240854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:25.303563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398e93a-ebaf-43db-8476-bcc3f9e12379 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:25.305667 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:25.374421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10fd6c-608d-488d-a388-c2bb5c052350 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:25.376499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:25.377361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.377646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.377927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.380673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:25.382422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:25.390473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 210/413] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:25 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:25.396268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:25.443573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97191588-8e4b-4216-85f1-6d72f53b696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:25.444502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97191588-8e4b-4216-85f1-6d72f53b696a None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.444786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97191588-8e4b-4216-85f1-6d72f53b696a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.445312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97191588-8e4b-4216-85f1-6d72f53b696a None admin] PATH_INFO: `/v3/projects/213a059b002f40c1bddeb6d9f1da1459/groups/7d93d0b9ae4844a8b8b3157e74ab0372/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.447320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97191588-8e4b-4216-85f1-6d72f53b696a None admin] RBAC: Authorizing `identity:create_grant(project_id=213a059b002f40c1bddeb6d9f1da1459, group_id=7d93d0b9ae4844a8b8b3157e74ab0372, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:25.457111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97191588-8e4b-4216-85f1-6d72f53b696a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:25.509200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d5ded-dd4f-4584-87e7-1c4b75e12213 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:25.545351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 204/414] 10.222.0.42 () {58 vars in 1442 bytes} [Thu May 26 21:07:25 2022] PUT /identity/v3/projects/213a059b002f40c1bddeb6d9f1da1459/groups/7d93d0b9ae4844a8b8b3157e74ab0372/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:25.550812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3253bce-6cac-4f4f-999b-4b7753113f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:25.552167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253bce-6cac-4f4f-999b-4b7753113f53 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.552645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253bce-6cac-4f4f-999b-4b7753113f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.558299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3253bce-6cac-4f4f-999b-4b7753113f53 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/groups/f8812941ebfb4a128bbb6888b1765cee/roles/c23de713167345639a2085add0a54ae6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.558299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253bce-6cac-4f4f-999b-4b7753113f53 None admin] RBAC: Authorizing `identity:create_grant(project_id=2dad1ade88dd4d0b8c70347b618248c9, group_id=f8812941ebfb4a128bbb6888b1765cee, role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:25.558299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3253bce-6cac-4f4f-999b-4b7753113f53 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:25.683441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 211/415] 10.222.0.42 () {58 vars in 1442 bytes} [Thu May 26 21:07:25 2022] PUT /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9/groups/f8812941ebfb4a128bbb6888b1765cee/roles/c23de713167345639a2085add0a54ae6 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:25.761226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd986-793c-4522-9038-b410c77fcabe None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.761761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd986-793c-4522-9038-b410c77fcabe None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.762222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcbd986-793c-4522-9038-b410c77fcabe None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.763774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 205/416] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:25.775705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af6f02-86a1-4066-bc0f-9b7c9ac3ba15 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:25.776020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af6f02-86a1-4066-bc0f-9b7c9ac3ba15 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:25.776305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af6f02-86a1-4066-bc0f-9b7c9ac3ba15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:25.799971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-56af6f02-86a1-4066-bc0f-9b7c9ac3ba15 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:25.801301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-56af6f02-86a1-4066-bc0f-9b7c9ac3ba15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:25.840070 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:25.891462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 212/417] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:25.915642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd986-793c-4522-9038-b410c77fcabe None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:25.947163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:26.039948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34d525-cdb2-4ab8-88f9-b2feddefa544 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:26.040477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34d525-cdb2-4ab8-88f9-b2feddefa544 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:26.040893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b34d525-cdb2-4ab8-88f9-b2feddefa544 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:26.042804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 213/418] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:26.054971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec957ce-a9fb-40b2-8841-4597ed97e77a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:26.055475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec957ce-a9fb-40b2-8841-4597ed97e77a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:26.055976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec957ce-a9fb-40b2-8841-4597ed97e77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:26.060772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcbd986-793c-4522-9038-b410c77fcabe None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:26.092613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-bec957ce-a9fb-40b2-8841-4597ed97e77a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:26.094125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bec957ce-a9fb-40b2-8841-4597ed97e77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:26.097284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:26.098404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:26.098871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:26.099350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:26.110991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:26.120661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:26.121609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:26.122496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:26.125568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 206/419] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:26.134596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:26.195119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 214/420] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:26.221182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec957ce-a9fb-40b2-8841-4597ed97e77a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:26.300835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:26.343737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eeb288-f36a-41b8-be50-9d3aafd227b9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:26.378158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:26.396492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:26.397769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:26.398943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:26.400017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:26.401177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:26.403383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec957ce-a9fb-40b2-8841-4597ed97e77a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:26.404013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:26.405620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:26.423394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 207/421] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:26.429380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:26.462271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:26.463124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:26.463413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:26.463694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:26.484616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:26.499598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:26.500555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:26.501348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 215/422] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:26.521091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:26.554424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d78088-a2dd-425c-8484-df147222cc83 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:26.603234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:26.606493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:26.608316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:26.609515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:26.646338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:26.723404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:26.724927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:26.797759 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c66ff-b6af-43bb-85cb-070247a9fdc4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:26.835258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 208/423] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 21:07:26.840836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:26.892141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:26.941032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:26.941714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:26.941996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:26.942269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:26.943028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:26.943336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:26.944758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:26.955708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad8759-db58-4035-b800-b87917bc9a77 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:26.974233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 216/424] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:26.984862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.000926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:27.001814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.002107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.002389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.003610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:27.005313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:27.007279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 209/425] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:26 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:27.013464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.018112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:27.122117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf0d9c-4da2-4003-bfb7-28bdbb278e15 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:27.136727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:27.160021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edceb5f1-da98-4a27-a1c7-71a94ba93a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:27.160966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceb5f1-da98-4a27-a1c7-71a94ba93a1b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.161258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceb5f1-da98-4a27-a1c7-71a94ba93a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.161536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceb5f1-da98-4a27-a1c7-71a94ba93a1b None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.162744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edceb5f1-da98-4a27-a1c7-71a94ba93a1b None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:27.164444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edceb5f1-da98-4a27-a1c7-71a94ba93a1b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:27.166076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 210/426] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:27 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:27.170120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae986bdd-d3ae-4d25-b2c6-6bed9be5c23a None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.170407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae986bdd-d3ae-4d25-b2c6-6bed9be5c23a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.170684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae986bdd-d3ae-4d25-b2c6-6bed9be5c23a None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.172744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 211/427] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:27.173481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae986bdd-d3ae-4d25-b2c6-6bed9be5c23a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.223977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6612a61d-2ac4-4c69-bd6c-ff9e0f0cff00 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:27.259017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:27.259675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.259999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.260278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.300884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae986bdd-d3ae-4d25-b2c6-6bed9be5c23a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:27.342956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9378f5b5-2e73-4dc9-a076-122840c94019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:27.343650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378f5b5-2e73-4dc9-a076-122840c94019 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.343959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378f5b5-2e73-4dc9-a076-122840c94019 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.344240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9378f5b5-2e73-4dc9-a076-122840c94019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.345080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378f5b5-2e73-4dc9-a076-122840c94019 None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=c23de713167345639a2085add0a54ae6, scope.project.id=a2cdddc85fc3414a8028b384e27eabad) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:27.345404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378f5b5-2e73-4dc9-a076-122840c94019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:27.346852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9378f5b5-2e73-4dc9-a076-122840c94019 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:27.347470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:27.349226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:27.360074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 212/428] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:27 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=c23de713167345639a2085add0a54ae6 => generated 611 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:27.361657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b59ec-f897-4d3c-b65c-c258363a024c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.361951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b59ec-f897-4d3c-b65c-c258363a024c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.362237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b59ec-f897-4d3c-b65c-c258363a024c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.365874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:27.395957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-350b59ec-f897-4d3c-b65c-c258363a024c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:27.397114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-350b59ec-f897-4d3c-b65c-c258363a024c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:27.488781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:27.502673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 217/429] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:27.513138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.563804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 213/430] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:27.608095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b59ec-f897-4d3c-b65c-c258363a024c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.669482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1b8f9-05c1-4f84-920d-eeb0b4d2590d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:27.687886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:27.715265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:27.716224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.716512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.716788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] PATH_INFO: `/v3/projects/a2cdddc85fc3414a8028b384e27eabad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.718196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] RBAC: Authorizing `identity:get_project(project_id=a2cdddc85fc3414a8028b384e27eabad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:27.720635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:27.723670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 218/431] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:27 2022] GET /identity/v3/projects/a2cdddc85fc3414a8028b384e27eabad => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:27.729675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.755354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b59ec-f897-4d3c-b65c-c258363a024c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:27.800400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:27.801508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.801796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.802077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.816038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:27.828950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:27.829891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 21:07:27.830707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:27.831623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 214/432] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:27.837796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.872689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bba22d8-7b39-4078-a526-80a5e69f5e92 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:27.884122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:27.920019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:27.920827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:27.921115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:27.921396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:27.922822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:27.924379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:27.926364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 219/433] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:27 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:27.932411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:27.957540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb33bf4-6a6e-4fa4-ab0a-82f659002991 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:27.992545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:27.999990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.000670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.000956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.001235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.006471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] RBAC: Adding query filter params (name=member) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:28.006471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:28.006471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:28.013127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 215/434] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:07:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:28.021063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:28.054235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89246b-28d9-4445-8340-d31e40dff997 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.091616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949df7c8-f52a-408b-ab86-85b73eb7a07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.092408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949df7c8-f52a-408b-ab86-85b73eb7a07b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.092709 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949df7c8-f52a-408b-ab86-85b73eb7a07b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.093045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949df7c8-f52a-408b-ab86-85b73eb7a07b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.093896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949df7c8-f52a-408b-ab86-85b73eb7a07b None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=a2cdddc85fc3414a8028b384e27eabad) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:28.094271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949df7c8-f52a-408b-ab86-85b73eb7a07b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:28.095764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949df7c8-f52a-408b-ab86-85b73eb7a07b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:28.109695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 220/435] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:27 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=a2cdddc85fc3414a8028b384e27eabad&role.id=1183a502796d46e580a92e3c20f04103 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:28.120822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:28.128898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f438a52-a3c3-4e40-b8fe-c938516c1a33 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.129526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f54bf-2aa3-4975-9254-65119b3d54ce None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.129851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f54bf-2aa3-4975-9254-65119b3d54ce None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.130139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f54bf-2aa3-4975-9254-65119b3d54ce None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.131931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 221/436] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:28.138219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bd3c-3c93-4b42-8a4f-0dd3edd22fc0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.138546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bd3c-3c93-4b42-8a4f-0dd3edd22fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.138784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c94bd3c-3c93-4b42-8a4f-0dd3edd22fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.157146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7c94bd3c-3c93-4b42-8a4f-0dd3edd22fc0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:28.158387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7c94bd3c-3c93-4b42-8a4f-0dd3edd22fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:28.170169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.171182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.171466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.171762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.255713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 222/437] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:28.268247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:28.270010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:28.280763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94bd3c-3c93-4b42-8a4f-0dd3edd22fc0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:28.329000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:28.385371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:28.399440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 216/438] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:28.401030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:28.404545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c94bd3c-3c93-4b42-8a4f-0dd3edd22fc0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.451092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a8e7d-bf90-4375-a4c4-201a3a80ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.452308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8e7d-bf90-4375-a4c4-201a3a80ae4b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.452654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8e7d-bf90-4375-a4c4-201a3a80ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.453002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a8e7d-bf90-4375-a4c4-201a3a80ae4b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.465252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a8e7d-bf90-4375-a4c4-201a3a80ae4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-598a8e7d-bf90-4375-a4c4-201a3a80ae4b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:28.474801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:28.475732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:28.477307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 223/439] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:28.477983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac071023-a0d7-4197-aaf0-b50fc71e6feb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.477983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac071023-a0d7-4197-aaf0-b50fc71e6feb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.478308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac071023-a0d7-4197-aaf0-b50fc71e6feb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.480264 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 224/440] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:28.482375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac071023-a0d7-4197-aaf0-b50fc71e6feb None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:28.545568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036075f-ee11-43d2-8014-3d475c018bd8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.586390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d4f47-8945-484e-a8dc-e9384d8599a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.587512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d4f47-8945-484e-a8dc-e9384d8599a6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.588480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d4f47-8945-484e-a8dc-e9384d8599a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.588480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d4f47-8945-484e-a8dc-e9384d8599a6 None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.590348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d4f47-8945-484e-a8dc-e9384d8599a6 None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:28.591894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d4f47-8945-484e-a8dc-e9384d8599a6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:28.593625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 217/441] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:28.595047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70092e-561b-48de-97b6-af37a9186a88 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.595336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70092e-561b-48de-97b6-af37a9186a88 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.595802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70092e-561b-48de-97b6-af37a9186a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.621983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac071023-a0d7-4197-aaf0-b50fc71e6feb None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.623622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:28.626167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ea70092e-561b-48de-97b6-af37a9186a88 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:28.627361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ea70092e-561b-48de-97b6-af37a9186a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:28.658495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.659438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.659877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.660161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.660922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:28.662278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:28.663795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:28.673052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 225/442] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:28.673895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:28.696751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:28.714714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 218/443] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:28.715008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70092e-561b-48de-97b6-af37a9186a88 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:28.799245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd5da0a-805d-40e8-a9c5-69feb69cd6a2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.821980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70092e-561b-48de-97b6-af37a9186a88 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.833719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.834582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.834866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.835136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.836352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:28.837788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:28.839428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 226/444] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:28.840219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:28.856625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.857276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.857637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.857853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.871204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:28.917161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:28.926599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:28.928080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:28.939118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d839c-39de-48e3-b621-e868500fb2dd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:28.973129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:28.973860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:28.974138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:28.974419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:28.984432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:28.992515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:28.993433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:28.994219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 227/445] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:28.994947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.032825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 219/446] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:29.033677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.090520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:29.098803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98858ad-8dc0-45b3-aeac-dbf9613ca3bc None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.123078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:29.131493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fdb11-d2b4-4ec0-a905-73004e138577 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.132290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.132944 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.133228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.133503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.134326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=1183a502796d46e580a92e3c20f04103, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:29.134645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.136712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.142708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 228/447] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:28 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=1183a502796d46e580a92e3c20f04103 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:29.143499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.175820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.176505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.176792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.177070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.177913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:29.178153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.179610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.194135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 220/448] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:29.195314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.262266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac6b3b-8fae-42b7-9326-0f3d6aef4952 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.299324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.300256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.300546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.300821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] PATH_INFO: `/v3/projects/213a059b002f40c1bddeb6d9f1da1459` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.302120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] RBAC: Authorizing `identity:get_project(project_id=213a059b002f40c1bddeb6d9f1da1459)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.303726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.305564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 229/449] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/projects/213a059b002f40c1bddeb6d9f1da1459 => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:29.311647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.316372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0189ed-4d54-400e-876d-11468fd62c63 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.341177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:29.359504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.360189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.360475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.360803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.391169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:29.428762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9cd862-6434-4bbb-8c2f-c626cfe5ce53 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.454173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.455649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.473855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.474722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.475069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.475347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] PATH_INFO: `/v3/groups/7d93d0b9ae4844a8b8b3157e74ab0372` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.477023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] RBAC: Authorizing `identity:get_group(group_id=7d93d0b9ae4844a8b8b3157e74ab0372)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.478477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.481688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 230/450] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/groups/7d93d0b9ae4844a8b8b3157e74ab0372 => generated 229 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:29.487449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.569950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 221/451] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:29.577244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.601433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:29.611162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ae38b-3c0f-433a-ad84-2ed71f6472c6 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.633184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:29.645260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.645915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.646200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.646516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.647347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] RBAC: Adding query filter params (group.id=7d93d0b9ae4844a8b8b3157e74ab0372, role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=213a059b002f40c1bddeb6d9f1da1459) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:29.647670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.649135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.655111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 231/452] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/role_assignments?group.id=7d93d0b9ae4844a8b8b3157e74ab0372&scope.project.id=213a059b002f40c1bddeb6d9f1da1459&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:29.678701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab9d7a-00ab-4f62-9ad8-eae0610cc603 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.723094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.724025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.724303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.724578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] PATH_INFO: `/v3/projects/2dad1ade88dd4d0b8c70347b618248c9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.725876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] RBAC: Authorizing `identity:get_project(project_id=2dad1ade88dd4d0b8c70347b618248c9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.727351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.729282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 222/453] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/projects/2dad1ade88dd4d0b8c70347b618248c9 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:29.735546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.840139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43fc19-fc91-4d7b-aa12-26d9edfeda82 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:29.876147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6102ecff-4ce6-4ed7-b3b3-b4e6b3aea199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:29.877001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102ecff-4ce6-4ed7-b3b3-b4e6b3aea199 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:29.878099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102ecff-4ce6-4ed7-b3b3-b4e6b3aea199 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:29.878099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6102ecff-4ce6-4ed7-b3b3-b4e6b3aea199 None admin] PATH_INFO: `/v3/groups/f8812941ebfb4a128bbb6888b1765cee` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:29.878706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102ecff-4ce6-4ed7-b3b3-b4e6b3aea199 None admin] RBAC: Authorizing `identity:get_group(group_id=f8812941ebfb4a128bbb6888b1765cee)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:29.880186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6102ecff-4ce6-4ed7-b3b3-b4e6b3aea199 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:29.881796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 232/454] 10.222.0.42 () {56 vars in 1180 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/groups/f8812941ebfb4a128bbb6888b1765cee => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:29.887651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:29.987211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49f854-3521-44c1-b7b4-23820ddc8afe None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:30.008916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:30.020704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810b959-068f-4f2d-9af1-ea00af64acde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:30.021307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b959-068f-4f2d-9af1-ea00af64acde None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:30.021591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b959-068f-4f2d-9af1-ea00af64acde None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:30.021869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810b959-068f-4f2d-9af1-ea00af64acde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:30.022696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810b959-068f-4f2d-9af1-ea00af64acde None admin] RBAC: Adding query filter params (group.id=f8812941ebfb4a128bbb6888b1765cee, role.id=c23de713167345639a2085add0a54ae6, scope.project.id=2dad1ade88dd4d0b8c70347b618248c9) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:30.023013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810b959-068f-4f2d-9af1-ea00af64acde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:30.024493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810b959-068f-4f2d-9af1-ea00af64acde None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:30.030249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 223/455] 10.222.0.42 () {56 vars in 1376 bytes} [Thu May 26 21:07:29 2022] GET /identity/v3/role_assignments?group.id=f8812941ebfb4a128bbb6888b1765cee&scope.project.id=2dad1ade88dd4d0b8c70347b618248c9&role.id=c23de713167345639a2085add0a54ae6 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:33.609936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2bff1-bc7e-4221-acc2-e872c165a62b None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.610475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2bff1-bc7e-4221-acc2-e872c165a62b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.610475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2bff1-bc7e-4221-acc2-e872c165a62b None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.612196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 233/456] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:33.628877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8b235-1ede-4e49-bfac-a874c76f434e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.629444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8b235-1ede-4e49-bfac-a874c76f434e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.629844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8b235-1ede-4e49-bfac-a874c76f434e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.663700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-35a8b235-1ede-4e49-bfac-a874c76f434e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:33.665732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-35a8b235-1ede-4e49-bfac-a874c76f434e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:33.732633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0674b7-adc1-414e-8a41-383730807258 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.733135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0674b7-adc1-414e-8a41-383730807258 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.733600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0674b7-adc1-414e-8a41-383730807258 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.735913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 234/457] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:33.752249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415bff-5293-4766-9ad7-2d6a165b5711 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.752737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415bff-5293-4766-9ad7-2d6a165b5711 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.753230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415bff-5293-4766-9ad7-2d6a165b5711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.805271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e2415bff-5293-4766-9ad7-2d6a165b5711 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:33.808189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e2415bff-5293-4766-9ad7-2d6a165b5711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:33.822036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 224/458] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:33.825374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb49b18-b60b-4573-b5a9-0b80c05b4abc None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.826246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb49b18-b60b-4573-b5a9-0b80c05b4abc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.826959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb49b18-b60b-4573-b5a9-0b80c05b4abc None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.829971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 225/459] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:33.832116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354286d-30a8-44cd-b9be-506fd1c1b971 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.832951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354286d-30a8-44cd-b9be-506fd1c1b971 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.833742 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354286d-30a8-44cd-b9be-506fd1c1b971 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.836608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 226/460] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:33.838810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00494-ae7a-45c5-9ddc-1ba2df81a313 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.839598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:33.841110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00494-ae7a-45c5-9ddc-1ba2df81a313 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.841788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d00494-ae7a-45c5-9ddc-1ba2df81a313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.868914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f9d00494-ae7a-45c5-9ddc-1ba2df81a313 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:33.870895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f9d00494-ae7a-45c5-9ddc-1ba2df81a313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:33.909118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 235/461] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:33.910420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5178e70-23ff-430d-90e4-639a00bb9115 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:33.910716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5178e70-23ff-430d-90e4-639a00bb9115 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:33.910996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5178e70-23ff-430d-90e4-639a00bb9115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:33.937929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e5178e70-23ff-430d-90e4-639a00bb9115 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:33.939106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e5178e70-23ff-430d-90e4-639a00bb9115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:33.943413 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 227/462] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:33.944637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d00494-ae7a-45c5-9ddc-1ba2df81a313 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:33.993445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.008785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 236/463] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:34.009650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5178e70-23ff-430d-90e4-639a00bb9115 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.063543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d00494-ae7a-45c5-9ddc-1ba2df81a313 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.098306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.100242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.101115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.101469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.101816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.112551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.113995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.115002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5178e70-23ff-430d-90e4-639a00bb9115 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:34.132428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:34.133312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:34.134227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:34.134227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:34.134227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.134227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:34.134227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 228/464] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:34.134504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.154169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.154973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.155255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.155529 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.166306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.168486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:34.187725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:34.188650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:34.189544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:34.189544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:34.189544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.189544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:34.193821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 237/465] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:34.194460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.243284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.247566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc97614-b2e6-4668-9ce4-720d5e8bb697 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.286399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.287474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.287771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.288058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.299215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.300674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.314920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e559e0-5247-4bdf-8cc9-ae8199c87ebe None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:34.319019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:34.320023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:34.321318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:34.321318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:34.321318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.321318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:34.321318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 229/466] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:34.321318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.356188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.356984 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.357273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.357621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.360697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.370473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.371972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:34.390716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:34.391569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:34.392461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:34.392461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:34.392461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:34.392461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:34.392461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 238/467] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:34.392733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.443295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39903765-6082-4151-b8d3-ebddee71ce46 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.482037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.482784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.483073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.483351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.484114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:34.484424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.485817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.499791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 230/468] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:34.499791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.512446 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.516131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d1280-94c4-4017-ac11-3ccae6ab639c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.556673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.557703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.558139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.559368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.559368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:34.559543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.560980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.573833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 239/469] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:34.574909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.613846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba0500-656b-4b01-adfc-cc3980412e89 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.627897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.650805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.652048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.652342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.652621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.653344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:34.653658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.655035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.666938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 231/470] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:34.667718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.684888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad875120-c650-4fdc-87cc-61f2921492d6 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.719500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.720190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.720544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.720821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.721612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:34.721847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.723217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.735010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 240/471] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:34.735857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:34.741283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.771525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a220efe-0e5a-49ec-ae5a-cb5bfc224a2d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.806114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5513cbe-2d4f-4f32-aa13-15705d2a63c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.806783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513cbe-2d4f-4f32-aa13-15705d2a63c0 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.807069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513cbe-2d4f-4f32-aa13-15705d2a63c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.807345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5513cbe-2d4f-4f32-aa13-15705d2a63c0 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.808101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5513cbe-2d4f-4f32-aa13-15705d2a63c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.809547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5513cbe-2d4f-4f32-aa13-15705d2a63c0 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.840379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2989102a-a7cf-4f77-bda9-e31446887dd0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:34.849363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:34.874012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:34.874742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.875021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.875320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.876047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:34.877485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:34.915917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 232/472] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:07:34 2022] POST /identity/v3/users => generated 249 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:34.917712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b23e0-70ef-4467-ac68-c0b36d5ffbeb None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:34.917996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b23e0-70ef-4467-ac68-c0b36d5ffbeb None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:34.918271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1b23e0-70ef-4467-ac68-c0b36d5ffbeb None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:34.919520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 233/473] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:34.920328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b23e0-70ef-4467-ac68-c0b36d5ffbeb None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.013616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 241/474] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:07:34 2022] POST /identity/v3/users => generated 246 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:35.014841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.078738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1b23e0-70ef-4467-ac68-c0b36d5ffbeb None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.113932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:35.125772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e66749-416f-4c0d-8c75-7f4b73d9d0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.126455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66749-416f-4c0d-8c75-7f4b73d9d0ad None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.126738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66749-416f-4c0d-8c75-7f4b73d9d0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.127011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e66749-416f-4c0d-8c75-7f4b73d9d0ad None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.128590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e66749-416f-4c0d-8c75-7f4b73d9d0ad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.130132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e66749-416f-4c0d-8c75-7f4b73d9d0ad None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.145018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f86cbb-18f4-414d-b7d2-fc380fedb7cb None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.185952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.186717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.186959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.187231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.187977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.189682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.260047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 234/475] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:07:34 2022] POST /identity/v3/users => generated 248 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:35.262115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5d9b2-1f6d-49df-9573-f066e57460d7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.262411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5d9b2-1f6d-49df-9573-f066e57460d7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.262702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5d9b2-1f6d-49df-9573-f066e57460d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.283330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-9bc5d9b2-1f6d-49df-9573-f066e57460d7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:35.285076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9bc5d9b2-1f6d-49df-9573-f066e57460d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:35.294575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 242/476] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:07:35 2022] POST /identity/v3/users => generated 248 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:35.373679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 235/477] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:35.397620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.425252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:35.527588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a060774a-b78f-4a75-81c4-eba3a1419b15 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.575869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.576074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.576325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.576512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.577280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:35.578691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:35.630296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.sql.core [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 21:07:35.630296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 21:07:35.630296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [parameters: {'id': 'ca201bfd04ec40eb93e2b6763245dfaf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 21:07:35.630296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80756) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result = self._query(query) May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application conn.query(q) May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result.read() May 26 21:07:35.648081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return ref.to_dict() May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 21:07:35.649015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application next(self.gen) May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application yield resource May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application next(self.gen) May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.session.rollback() May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.force_reraise() May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise self.value May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application session.commit() May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 21:07:35.649962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._prepare_impl() May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.session.flush() May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._flush(objects) May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application compat.raise_( May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application flush_context.execute() May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rec.execute(self) May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 21:07:35.650833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 21:07:35.653714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result = self._query(query) May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application conn.query(q) May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result.read() May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 21:07:35.655371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [parameters: {'id': 'ca201bfd04ec40eb93e2b6763245dfaf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:35.656513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 21:07:35.663996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:35.665196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 243/478] 10.222.0.42 () {60 vars in 1131 bytes} [Thu May 26 21:07:35 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:35.665283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5d9b2-1f6d-49df-9573-f066e57460d7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:35.835333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5d9b2-1f6d-49df-9573-f066e57460d7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:35.858916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:35.886667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2455a9-5147-40d1-9ced-a812d23225a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:35.887489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2455a9-5147-40d1-9ced-a812d23225a3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:35.887784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2455a9-5147-40d1-9ced-a812d23225a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:35.888110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2455a9-5147-40d1-9ced-a812d23225a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:35.908691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2455a9-5147-40d1-9ced-a812d23225a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-7f2455a9-5147-40d1-9ced-a812d23225a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:35.920647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:35.921554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:35.922382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 236/479] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:07:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:35.927536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:36.085462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:36.107863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7507daa2-5b0a-4f27-a6a6-936d0244e519 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:36.168281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f1270-df1b-4bf8-8c3d-372faf7e7674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:36.169350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f1270-df1b-4bf8-8c3d-372faf7e7674 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:36.169839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f1270-df1b-4bf8-8c3d-372faf7e7674 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:36.170337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f1270-df1b-4bf8-8c3d-372faf7e7674 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:36.171300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f1270-df1b-4bf8-8c3d-372faf7e7674 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:36.171845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f1270-df1b-4bf8-8c3d-372faf7e7674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:36.177260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577f1270-df1b-4bf8-8c3d-372faf7e7674 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:36.187193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 244/480] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:37.734500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1e11a-3199-40b8-91fd-c9a9690aaa57 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:37.737179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1e11a-3199-40b8-91fd-c9a9690aaa57 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:37.737179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd1e11a-3199-40b8-91fd-c9a9690aaa57 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:37.737643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 237/481] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:37.743326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d10be-65f3-4175-a53e-16e53c3f7726 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:37.743810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d10be-65f3-4175-a53e-16e53c3f7726 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:37.744107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d10be-65f3-4175-a53e-16e53c3f7726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:37.766238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-946d10be-65f3-4175-a53e-16e53c3f7726 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:37.767439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-946d10be-65f3-4175-a53e-16e53c3f7726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:37.810880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9a0c-64d1-4924-b816-54a701ab59c4 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:37.811217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9a0c-64d1-4924-b816-54a701ab59c4 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:37.811481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a9a0c-64d1-4924-b816-54a701ab59c4 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:37.813237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 238/482] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:37.825117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d72da-2a93-4037-98ae-07c2d5e69c12 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:37.825742 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d72da-2a93-4037-98ae-07c2d5e69c12 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:37.826226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d72da-2a93-4037-98ae-07c2d5e69c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:37.851865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-492d72da-2a93-4037-98ae-07c2d5e69c12 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:37.853406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-492d72da-2a93-4037-98ae-07c2d5e69c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:37.879773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 245/483] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:37.909867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d10be-65f3-4175-a53e-16e53c3f7726 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:37.976309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 239/484] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:37.999600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d72da-2a93-4037-98ae-07c2d5e69c12 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.036869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:38.059120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d10be-65f3-4175-a53e-16e53c3f7726 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.106142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f8484e-8836-4e4e-90c8-bbfe5a2cdcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.106965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8484e-8836-4e4e-90c8-bbfe5a2cdcb8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.107247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8484e-8836-4e4e-90c8-bbfe5a2cdcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.107560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f8484e-8836-4e4e-90c8-bbfe5a2cdcb8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.129784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f8484e-8836-4e4e-90c8-bbfe5a2cdcb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a7f8484e-8836-4e4e-90c8-bbfe5a2cdcb8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:38.138581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:38.139611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 246/485] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:38.144376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c45d8-8d7f-4e4f-8e4b-6ebfafdf10fa None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.145255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c45d8-8d7f-4e4f-8e4b-6ebfafdf10fa None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.145255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c45d8-8d7f-4e4f-8e4b-6ebfafdf10fa None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.145255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 247/486] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:38.148931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090f1d3-46ce-4ce4-854a-3904af4d9f4f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.149291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090f1d3-46ce-4ce4-854a-3904af4d9f4f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.149644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090f1d3-46ce-4ce4-854a-3904af4d9f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.168634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4090f1d3-46ce-4ce4-854a-3904af4d9f4f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:38.169941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4090f1d3-46ce-4ce4-854a-3904af4d9f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:38.186601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d72da-2a93-4037-98ae-07c2d5e69c12 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.188307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:38.225209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.226197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.226709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.227046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.239298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:38.245574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 248/487] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:38.246552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090f1d3-46ce-4ce4-854a-3904af4d9f4f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:38.248331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:38.249216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:38.249989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:38.250754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 240/488] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:38.256367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.316558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:38.353563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090f1d3-46ce-4ce4-854a-3904af4d9f4f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.382137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aaa0d8-611c-4137-9e9f-f58d1b5ade99 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.393208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.393881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.394164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.394440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.395191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] RBAC: Adding query filter params (name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:38.395495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:38.396924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:38.409741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 249/489] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:38.409741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.423730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981524e7-270b-40e2-b07d-71708d2c619a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.424717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981524e7-270b-40e2-b07d-71708d2c619a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.425169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981524e7-270b-40e2-b07d-71708d2c619a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.425590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981524e7-270b-40e2-b07d-71708d2c619a None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.426528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981524e7-270b-40e2-b07d-71708d2c619a None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:38.426845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981524e7-270b-40e2-b07d-71708d2c619a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:38.428238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981524e7-270b-40e2-b07d-71708d2c619a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:38.437617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 241/490] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:38.439094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b909771-f749-483b-8ec1-d81fca339713 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.439388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b909771-f749-483b-8ec1-d81fca339713 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.439665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b909771-f749-483b-8ec1-d81fca339713 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.443788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 242/491] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:38.443788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b909771-f749-483b-8ec1-d81fca339713 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.468202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:38.528250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093118d-e0a2-4a33-8cdc-5fcd35aa4644 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.549699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b909771-f749-483b-8ec1-d81fca339713 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.558489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:38.567276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.577339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.577339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.577339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.581116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:38.588304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4645c44-3b2d-4524-8002-fb6cb27f2d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:38.589729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:38.590672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:38.591921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:38.593050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645c44-3b2d-4524-8002-fb6cb27f2d1d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.593050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645c44-3b2d-4524-8002-fb6cb27f2d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.593050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4645c44-3b2d-4524-8002-fb6cb27f2d1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.593405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 250/492] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:38.593405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.607907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645c44-3b2d-4524-8002-fb6cb27f2d1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:38.609722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4645c44-3b2d-4524-8002-fb6cb27f2d1d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-b4645c44-3b2d-4524-8002-fb6cb27f2d1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:38.628967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:38.630000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:38.630847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:38.630847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:38.630847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:38.630847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:38.630847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 243/493] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:38.631694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c0b9-84db-42ab-abe3-13ced2906f1c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.631827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c0b9-84db-42ab-abe3-13ced2906f1c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.632041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c0b9-84db-42ab-abe3-13ced2906f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.650234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7170c0b9-84db-42ab-abe3-13ced2906f1c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:38.652192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7170c0b9-84db-42ab-abe3-13ced2906f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:38.783967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49e708-0ccb-4317-bdcc-933af2115e85 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.785593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 244/494] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:38.786883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c0b9-84db-42ab-abe3-13ced2906f1c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.807283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:38.827921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.828761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.829028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.829304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.840790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:38.842279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:38.861000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:38.861920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:38.862764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:38.862764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:38.862764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:38.862764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:38.862764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 251/495] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:38.863025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.916261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c0b9-84db-42ab-abe3-13ced2906f1c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:38.916782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:38.953156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:38.953866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:38.954109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:38.954400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:38.955361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:38.955672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:38.957081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:38.965921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 245/496] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:38.970530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:38.982245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8048bb-ad81-41bd-b2b5-0b688662cbcd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.020225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.021063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.021346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.021624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.022383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:39.022695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.024152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.043108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 252/497] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:39.043108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.051666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.088073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c7ac71-b3fd-4b77-9a03-231fd6ceab4e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.124351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.125295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.125588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.125869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.136962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:39.142271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:39.144973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:39.145847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:39.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:39.157625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031d89-bc45-4026-88ba-42b9ffcf125a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.157710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 246/498] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:39.157710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.189908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.190573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.190858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.191137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.191897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:39.192209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.193588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.205934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 253/499] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:39.206629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.259997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f6de6-bcdc-4ade-b050-7d1bee4208ef None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.288172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.299997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.299997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.299997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.299997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.305920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a9a50-9590-4267-a5d0-a89ebe58215c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.306925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.308927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:39.327293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:39.328347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:39.329394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:39.329394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:39.329394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:39.329394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:39.330905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 247/500] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:39.331685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.341419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca148c0-1e13-45ec-a9ab-dc5241187e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.342216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca148c0-1e13-45ec-a9ab-dc5241187e60 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.342504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca148c0-1e13-45ec-a9ab-dc5241187e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.342780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca148c0-1e13-45ec-a9ab-dc5241187e60 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.350454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca148c0-1e13-45ec-a9ab-dc5241187e60 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.351941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca148c0-1e13-45ec-a9ab-dc5241187e60 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-fca148c0-1e13-45ec-a9ab-dc5241187e60 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:39.368260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:39.369434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 254/501] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:39.370769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1804-1689-4fda-b1c6-d7803f9b8ddc None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.371534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1804-1689-4fda-b1c6-d7803f9b8ddc None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.371534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca1804-1689-4fda-b1c6-d7803f9b8ddc None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.373151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 255/502] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:39.373461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca1804-1689-4fda-b1c6-d7803f9b8ddc None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.395654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.441316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b9a77e-1889-4eb7-94ce-c6a4ea05cc6a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.477318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.477985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.478269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.478545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.479990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:39.480350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.480582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca1804-1689-4fda-b1c6-d7803f9b8ddc None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.481776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.489955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 248/503] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:39.490738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.518385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.519176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.519459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.519734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.527764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.529238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.535254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:39.543981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:39.547601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 256/504] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:39.548713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.597868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a5694d-d137-441c-9641-a945aa9fc6a9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.631643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a5078b-9a2f-4334-8cf9-5ee4e043e799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.631853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.632333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5078b-9a2f-4334-8cf9-5ee4e043e799 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.632619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5078b-9a2f-4334-8cf9-5ee4e043e799 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.632899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a5078b-9a2f-4334-8cf9-5ee4e043e799 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.633667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5078b-9a2f-4334-8cf9-5ee4e043e799 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:39.633956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5078b-9a2f-4334-8cf9-5ee4e043e799 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.635329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a5078b-9a2f-4334-8cf9-5ee4e043e799 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.647648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 249/505] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:39.648892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9baf5-88b5-42db-8f23-2ae43f50af29 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.649188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9baf5-88b5-42db-8f23-2ae43f50af29 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.649467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f9baf5-88b5-42db-8f23-2ae43f50af29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.651863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1827881c-7e66-487a-ae67-d1b33e999117 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.665666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-18f9baf5-88b5-42db-8f23-2ae43f50af29 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:39.667385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-18f9baf5-88b5-42db-8f23-2ae43f50af29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:39.686609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.687271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.687550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.687857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.688687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:39.688995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.690433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.727869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 257/506] 10.222.0.42 () {56 vars in 1139 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:39.728710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.743878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 250/507] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:39.744949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9baf5-88b5-42db-8f23-2ae43f50af29 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.775457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.844206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475ab94-6254-48a2-88cc-fda4c3b48c6a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.844779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f9baf5-88b5-42db-8f23-2ae43f50af29 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:39.879399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.879591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:39.880019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.880133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:39.880303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.880424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:39.880574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.880662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:39.881450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:39.881714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.883156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.888075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:39.891967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:39.893428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:39.911626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:39.912711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:39.913633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:39.913633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:39.913633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:39.913633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:39.913633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 258/508] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:39.913898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:39.921581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 251/509] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:39.922520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.011717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.015362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c67868c-9c1b-434c-9aaa-161a42273594 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.020876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c94780-eab9-49e2-9e95-91049596048a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.049743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.050534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.050811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.051086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.055134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.055807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.056095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.056371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.058642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.060136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:40.075137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:40.076343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 259/510] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:40.082219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.125011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.126491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.127643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.177618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843dd6e9-7fcf-4711-ae8d-abd9c4a72d26 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.212190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.212992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.213269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.213571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.215715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.223974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.225439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.226093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 252/511] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:40.226997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:40.245431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:40.246339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:40.247177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:40.247177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:40.247177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:40.247177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:40.247177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 260/512] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:40.247892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.328883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba81c3-1f2d-4b67-99bf-ad43249bcc83 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.346648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c6f07-7937-4668-829d-1b49b08dcb38 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.348293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.370949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.371615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.371968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.372217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.372937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:40.373247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.374625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.381090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.381733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.382012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.382282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.386341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 253/513] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:40.387131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.437375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.451967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.453434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.488634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205459a6-3776-4b71-afd4-d6b212f2b31d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.523248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.523955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.524241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.524519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.525359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:40.525674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.527704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.545835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 254/514] 10.222.0.42 () {56 vars in 1137 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:40.546643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.559761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 261/515] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:40.560629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.565817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.654663 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25210950-9908-407c-a05d-c6deff6e8270 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.664521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.667375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2971bc-e94a-45ca-b634-b1c965108de2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.689536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.690257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.690557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.690823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.701447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.702923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.714333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.715000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.715280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.715551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.716325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:40.716643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.718002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.730283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 262/516] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:40.731099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:40.734822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:40.736011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:40.737062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:40.737062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:40.737062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:40.737062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:40.737426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 255/517] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:40.738841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.801233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.840249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62b6ff-7aae-4fe6-919a-781e84b8f82d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.851798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77a12-ce25-4f9a-90e6-94011e56dd43 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:40.874421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.875208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.875485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.875765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.882742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.884240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:40.887484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:40.888172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:40.888456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:40.888734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:40.899600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:40.900555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:40.901471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 263/518] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:40.902252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:40.921373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:40.962891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:40.964853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.006508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a507b-32ad-4cc8-8b76-3df48aefb606 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.029470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.040505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.041293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.043519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.043519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.052455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.053908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.069066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 256/519] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:41.069893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:41.072089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:41.073083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:41.073911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:41.073911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:41.073911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:41.073911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:41.073911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 264/520] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:41.074797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.132490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.172542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac67c7-c1be-4878-b789-9fa07c0e6150 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.173348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603931a-c7bc-4be9-903d-4d4dacf1eecd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.207100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.207785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.208068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.208345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.208478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.209078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.209183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.209468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.209741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.210460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:41.210554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.210768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.212218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.224089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 265/521] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:41.224849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.258820 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.316407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 257/522] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:07:41 2022] POST /identity/v3/users => generated 247 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:07:41.323410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4906a4e-69d7-44d5-bd75-77d80f454008 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.327139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.364497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.365182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.365465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.365751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.366704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:41.367019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.368611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.391011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 266/523] 10.222.0.42 () {56 vars in 1137 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:41.391921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.453766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06025f78-4161-4087-9de6-955c49159cb3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.473201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.495895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.495895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.495895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.495895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.504529 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.506744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.508273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.514584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bfab2-cb3f-457e-bfb7-a63ecbcf2ecd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:41.528530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:41.530139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:41.531135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:41.531135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:41.531135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:41.531135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:41.531135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 258/524] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:41.531135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.563373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.564086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.564377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.564718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.565383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:41.565704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.567089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.583054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 267/525] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:41.583950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.648024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b565c3-b7f8-454e-a0b0-1162d8a9fa1b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.686757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.688024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.688379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.688724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.690902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26242bf-036a-4caa-94cb-34f682e167b7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.696509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.698135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:41.717535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:41.718593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:41.719498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 259/526] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:41.720150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.744240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.745447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.745726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.746063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.753775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.769668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:41.816449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.818123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.830044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a5142-60b7-4200-af58-61a608091634 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:41.871141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:41.872202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:41.872484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:41.872869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:41.873594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:41.873904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:41.875883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:41.895703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 260/527] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:41.895703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.925281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 268/528] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:41.926139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:41.983374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.001284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.013443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce6c95-7e6a-45ae-b05c-9eb073bc1cf6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.033824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d841d13-aed3-4166-be08-5db9b59ffcc5 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.048592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.049495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.049779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.050057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.058682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.060167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.070369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.071370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.071720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.072118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.073494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:42.073866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:42.075190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:42.076092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:42.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:42.082381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.082468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 261/529] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:42.082468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.101543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 269/530] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:42.102521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.199437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfc796c-284c-479b-bdb2-8bdef2cb4bb4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.214702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a09eb8-b379-4be2-8d2f-2c351421e03f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.227737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.233939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.234749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.235031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.235310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.242100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.243553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.249706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.250549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.250827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.251099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:42.258386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:42.259198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 262/531] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:42.261067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.266568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.268072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.272005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:42.286718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:42.287729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:42.288650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:42.288650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:42.288650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:42.288650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:42.288650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 270/532] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:42.289039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.370399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf54be-d692-4216-a414-8419a08907e2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.402307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f13f0-2706-48f4-bd1a-aba5b2a217a4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.411418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.412319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.412430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.412666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.413456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:42.413771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.415256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.433972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 263/533] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:42.433972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.443450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.444147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.444434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.444713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.445812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=e757ff94b7474fd99a1c7221be19c64d) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:42.446133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.447590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.454141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 271/534] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/role_assignments?user.id=e757ff94b7474fd99a1c7221be19c64d&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:42.454931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.512377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.529149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.568249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c08790-3a7f-466f-9411-321bf611633e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.609602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.610650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.611118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.611591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.612629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:42.613139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.614808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.623256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e14b-c834-4a93-bb32-42bcd8884e81 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.629858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 264/535] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:42.631105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.681017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.681795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.682077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.682968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.683707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:42.684157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.685596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.698528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 272/536] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:42.702773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.782512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22f275-ffe1-498d-bc69-66ead4ac1a86 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.811078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.817119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:42.826111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59685d-d6fb-4707-b8b0-c348cd8b7809 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:42.827817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.829518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.829953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.830372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.831348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=eefd00457d354271867932d336d7f69c) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:42.831871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.833921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.843510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 265/537] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/role_assignments?user.id=eefd00457d354271867932d336d7f69c&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:42.844973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:42.881782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:42.882832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:42.883264 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:42.883733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:42.884806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=a98415ce308d41f6b8d5db6ca3f2ff19) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:42.885327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:42.887186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:42.894817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 273/538] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/role_assignments?user.id=a98415ce308d41f6b8d5db6ca3f2ff19&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:43.029338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed5123b-48f5-4e48-9a21-9aac37cd8f75 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:43.083653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:43.084408 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:43.084689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:43.084963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:43.092838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:43.094311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:43.110182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:43.111080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:43.112011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 266/539] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:43.116962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:43.228949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eb6ef-b9d0-40ab-9f38-3963656d6bca None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:43.275221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de03bb-53d7-4ee7-b76b-47dbbc632739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:43.275905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de03bb-53d7-4ee7-b76b-47dbbc632739 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:43.276843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de03bb-53d7-4ee7-b76b-47dbbc632739 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:43.277118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de03bb-53d7-4ee7-b76b-47dbbc632739 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:43.277906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de03bb-53d7-4ee7-b76b-47dbbc632739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:43.278212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de03bb-53d7-4ee7-b76b-47dbbc632739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:43.279659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de03bb-53d7-4ee7-b76b-47dbbc632739 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:43.297199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 274/540] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:43.303040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:43.374428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:43.443920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6259bdef-ecbd-4fd7-8852-1b641173a475 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:43.491903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003b1127-95fb-463f-9764-3b790b14aa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:43.492568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b1127-95fb-463f-9764-3b790b14aa3b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:43.492930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b1127-95fb-463f-9764-3b790b14aa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:43.493208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003b1127-95fb-463f-9764-3b790b14aa3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:43.494048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b1127-95fb-463f-9764-3b790b14aa3b None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=b6cf996e1fdb4a458d06bfd23d082dec) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:43.494359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b1127-95fb-463f-9764-3b790b14aa3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:43.495903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003b1127-95fb-463f-9764-3b790b14aa3b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:43.502302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 267/541] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:43 2022] GET /identity/v3/role_assignments?user.id=b6cf996e1fdb4a458d06bfd23d082dec&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:43.649423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda344e-52d7-43ee-b051-5547a1732a51 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:43.649761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda344e-52d7-43ee-b051-5547a1732a51 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:43.649932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda344e-52d7-43ee-b051-5547a1732a51 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:43.652175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 275/542] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:43.657666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9b73-83f1-4e64-9324-e4d6be0e4242 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:43.657960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9b73-83f1-4e64-9324-e4d6be0e4242 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:43.658239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae9b73-83f1-4e64-9324-e4d6be0e4242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:43.684870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-cbae9b73-83f1-4e64-9324-e4d6be0e4242 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:43.686088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-cbae9b73-83f1-4e64-9324-e4d6be0e4242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:43.781222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 268/543] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:43.811575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda344e-52d7-43ee-b051-5547a1732a51 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:43.827163 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:43.946003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda344e-52d7-43ee-b051-5547a1732a51 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:43.993914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:43.995292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:43.995568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:43.995846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:44.016809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:44.024946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:44.025875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:44.026762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 276/544] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:44.032027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9b73-83f1-4e64-9324-e4d6be0e4242 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:44.088356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:44.142549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae9b73-83f1-4e64-9324-e4d6be0e4242 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:44.177771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:44.178442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:44.178728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:44.179005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:44.179769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:44.180135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:44.181501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:44.190187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 269/545] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:44.200899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:44.284878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:44.326164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd68e19-c39a-4a0d-8932-b6681daec9ee None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:44.363988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:44.364790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:44.365072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:44.365346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:44.376328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:44.377793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:44.395619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:44.396517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:44.397773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:44.397773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:44.397773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:44.397773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:44.397773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 277/546] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:44.405113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:44.578202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130f5f6-f1b7-4d8f-b46a-a96a3cd12e7d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:44.595492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:44.621286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:44.621956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:44.622237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:44.622508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:44.623294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:44.624061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:44.625448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:44.638395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 270/547] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:44.648829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:44.817120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d871e4b1-e7a0-4267-9759-119b6d386056 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:44.854722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:44.868435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:44.869243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:44.869530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:44.869836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:44.878005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:44.879462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:44.902004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:44.903174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:44.904344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 278/548] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:07:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:44.908842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:45.068282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e02cb5-47e6-4a1e-8ba5-7eb891058fae None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:45.109130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a126567-b00b-4519-a74e-ded2472f5116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:45.109722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126567-b00b-4519-a74e-ded2472f5116 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.110004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126567-b00b-4519-a74e-ded2472f5116 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.110280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a126567-b00b-4519-a74e-ded2472f5116 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.111122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126567-b00b-4519-a74e-ded2472f5116 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:45.111429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126567-b00b-4519-a74e-ded2472f5116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:45.112907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a126567-b00b-4519-a74e-ded2472f5116 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:45.132154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 271/549] 10.222.0.42 () {56 vars in 1135 bytes} [Thu May 26 21:07:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:45.138407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:45.282256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75d49-41ee-4933-82d4-8aaebdd8fbef None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.282603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75d49-41ee-4933-82d4-8aaebdd8fbef None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.282815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75d49-41ee-4933-82d4-8aaebdd8fbef None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.285744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 272/550] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:45.290603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4d97e-d124-4594-a1ee-94a58ee31f91 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.290887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4d97e-d124-4594-a1ee-94a58ee31f91 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.291163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa4d97e-d124-4594-a1ee-94a58ee31f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.307571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e20d3-abdb-475e-bf61-db54273a934b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:45.311468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-9fa4d97e-d124-4594-a1ee-94a58ee31f91 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:45.312725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9fa4d97e-d124-4594-a1ee-94a58ee31f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:45.344144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:45.394133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 273/551] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:45.408521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05454f3a-498e-44a6-b142-16f0987cd32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:45.409503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05454f3a-498e-44a6-b142-16f0987cd32e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.410013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05454f3a-498e-44a6-b142-16f0987cd32e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.410490 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05454f3a-498e-44a6-b142-16f0987cd32e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.412460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4d97e-d124-4594-a1ee-94a58ee31f91 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:45.525510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:45.532714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa4d97e-d124-4594-a1ee-94a58ee31f91 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:45.571251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:45.572105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.572444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.572750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.585057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:45.588673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05454f3a-498e-44a6-b142-16f0987cd32e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:45.590210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05454f3a-498e-44a6-b142-16f0987cd32e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:45.593352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:45.594379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:45.595198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:45.600439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 274/552] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:45.601725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:45.664193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:45.730428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2e6f3-65cd-4a29-a15d-0ea64408f8c2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:45.766183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 279/553] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:45.767806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781cdf7-1189-48ac-abf9-68ca30e96c40 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.768095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d1529-982d-48f6-a7df-6599ced57d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:45.768217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781cdf7-1189-48ac-abf9-68ca30e96c40 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.768503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3781cdf7-1189-48ac-abf9-68ca30e96c40 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.769232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1529-982d-48f6-a7df-6599ced57d63 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.769692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1529-982d-48f6-a7df-6599ced57d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.774169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 280/554] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:45.774169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781cdf7-1189-48ac-abf9-68ca30e96c40 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:45.774803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d1529-982d-48f6-a7df-6599ced57d63 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.775973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d1529-982d-48f6-a7df-6599ced57d63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:45.776997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d1529-982d-48f6-a7df-6599ced57d63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:45.778626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d1529-982d-48f6-a7df-6599ced57d63 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:45.788253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 275/555] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:45.789800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63601169-1f32-4b56-8c87-ceca089648b1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.790294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63601169-1f32-4b56-8c87-ceca089648b1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.790840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63601169-1f32-4b56-8c87-ceca089648b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.809292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-63601169-1f32-4b56-8c87-ceca089648b1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:45.810951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-63601169-1f32-4b56-8c87-ceca089648b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:45.888010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:45.892170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 276/556] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:45.893297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63601169-1f32-4b56-8c87-ceca089648b1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:45.907013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3781cdf7-1189-48ac-abf9-68ca30e96c40 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:45.918138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:45.948566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:45.949364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:45.949653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:45.949929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:45.962585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:45.965195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:45.987953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:45.988906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:45.989788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:45.989788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:45.989788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:45.989788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:45.989788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 281/557] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:45.990527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.031392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63601169-1f32-4b56-8c87-ceca089648b1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.068369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.069172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.069455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.069726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.081248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.082763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:46.101563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:46.102967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:46.103887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:46.103887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:46.103887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:46.103887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:46.103887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 277/558] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:46.105299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.114357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422b99b-5048-46bb-b772-6cc57407d9be None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.151726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc749db8-c15e-4eda-b872-671c4fb83f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.152717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749db8-c15e-4eda-b872-671c4fb83f70 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.153005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749db8-c15e-4eda-b872-671c4fb83f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.153664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc749db8-c15e-4eda-b872-671c4fb83f70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.157496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.167660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.168980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc749db8-c15e-4eda-b872-671c4fb83f70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-cc749db8-c15e-4eda-b872-671c4fb83f70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:46.176074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:46.177012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:46.178608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 282/559] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:46.180941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf7202-865f-4b86-b88a-21c5d6101b2f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.181443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf7202-865f-4b86-b88a-21c5d6101b2f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.182054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcf7202-865f-4b86-b88a-21c5d6101b2f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.183561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 283/560] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:46.184238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf7202-865f-4b86-b88a-21c5d6101b2f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.211386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2713529-99f4-4004-8bc4-fa8267b0df0c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.246451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.247111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.247392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.247665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.248415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:46.248724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.250070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.261775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 278/561] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:46.262572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.292597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcf7202-865f-4b86-b88a-21c5d6101b2f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.327332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379447e-7fbc-41f3-931b-083f93c4ef8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.328071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379447e-7fbc-41f3-931b-083f93c4ef8b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.328340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379447e-7fbc-41f3-931b-083f93c4ef8b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.328618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379447e-7fbc-41f3-931b-083f93c4ef8b None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.329375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379447e-7fbc-41f3-931b-083f93c4ef8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:46.329686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379447e-7fbc-41f3-931b-083f93c4ef8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.331062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379447e-7fbc-41f3-931b-083f93c4ef8b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.345022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 284/562] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:46.346277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610cfac-71a0-424c-a11a-0f27d70091fc None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.346564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610cfac-71a0-424c-a11a-0f27d70091fc None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.346844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610cfac-71a0-424c-a11a-0f27d70091fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.363026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7610cfac-71a0-424c-a11a-0f27d70091fc None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:46.364244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7610cfac-71a0-424c-a11a-0f27d70091fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:46.368089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bda2cc-9003-47fe-87a1-af7125583c09 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.388316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.402510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3afd68-b17b-4836-8fa5-87ffdcba9559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.403216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3afd68-b17b-4836-8fa5-87ffdcba9559 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.403632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3afd68-b17b-4836-8fa5-87ffdcba9559 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.403952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3afd68-b17b-4836-8fa5-87ffdcba9559 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.404710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3afd68-b17b-4836-8fa5-87ffdcba9559 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:46.405027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3afd68-b17b-4836-8fa5-87ffdcba9559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.406404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3afd68-b17b-4836-8fa5-87ffdcba9559 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.414495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.414662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 279/563] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:46.416089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def968e-00ff-4589-8d63-a8bc719a472d None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.416385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def968e-00ff-4589-8d63-a8bc719a472d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.417200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def968e-00ff-4589-8d63-a8bc719a472d None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.418468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 280/564] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:46.419217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def968e-00ff-4589-8d63-a8bc719a472d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.434869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 285/565] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:46.435764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610cfac-71a0-424c-a11a-0f27d70091fc None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.524569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def968e-00ff-4589-8d63-a8bc719a472d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.544615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610cfac-71a0-424c-a11a-0f27d70091fc None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.559149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.559979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.560266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.560547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.567516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.568993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.582454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985203ec-d2e3-4540-86ad-84f764ea0b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.583254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985203ec-d2e3-4540-86ad-84f764ea0b1b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.583529 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985203ec-d2e3-4540-86ad-84f764ea0b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.583826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985203ec-d2e3-4540-86ad-84f764ea0b1b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:46.583920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:46.584840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:46.585713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 281/566] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:46.586188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.596292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985203ec-d2e3-4540-86ad-84f764ea0b1b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.597771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985203ec-d2e3-4540-86ad-84f764ea0b1b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-985203ec-d2e3-4540-86ad-84f764ea0b1b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:46.612928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:46.613897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:46.614823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 286/567] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:46.615416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae867d-21e9-41e3-b36d-aad6e4147db3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.615633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae867d-21e9-41e3-b36d-aad6e4147db3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.616173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae867d-21e9-41e3-b36d-aad6e4147db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.632363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-64ae867d-21e9-41e3-b36d-aad6e4147db3 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:46.633557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-64ae867d-21e9-41e3-b36d-aad6e4147db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:46.639033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.728327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.750645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8202b14-8073-4f24-ae08-7bb95bab2c02 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.754375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 287/568] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:46.755527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae867d-21e9-41e3-b36d-aad6e4147db3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.787645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.788471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.788759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.789034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.799570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.801067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:46.818660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:46.819574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:46.820799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:46.820799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:46.820799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:46.820799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:46.820799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 282/569] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:46.821087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.873791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae867d-21e9-41e3-b36d-aad6e4147db3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.909165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.909966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.910248 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.910521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.914523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.921081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:46.921670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74441ff-4bb1-4bbf-9b94-d4f79491ee5d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:46.928891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:46.930012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:46.931053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 288/570] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:46.931908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:46.957738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:46.958324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:46.958610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:46.958883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:46.959668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:46.960008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:46.961433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:46.966753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:46.973349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 283/571] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:46.974142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.029466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c8e79-386e-4c08-b00e-16c970615513 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.063011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.063681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.063993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.064268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.065106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:47.065417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.066877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.075579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8558728f-0089-478d-9565-0368a4a1d1bc None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.084695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 289/572] 10.222.0.42 () {56 vars in 1139 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:47.085502 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.110315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.111010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.111294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.111570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.121781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:47.129773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:47.130730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:47.131560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 284/573] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:47.132420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.152619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.182723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.189202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d732d7-511a-4cda-b890-4ccb066ebe9f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.223225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.223838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.224117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.224389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.225108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:47.225412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.226894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.237242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03c9ad-d4a3-41e8-a0a2-a2b1f219d320 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.239562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 290/574] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:47.240487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.272132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.272799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.273082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.273362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.274219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:47.274536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.275967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.283907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 285/575] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:47.284710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.345677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526e985-687d-4e8c-abb8-763f1114caf6 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.359807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.379193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.379931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.380209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.380483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.388518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=ededf6d88dee4745978c8566940b2180) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:47.388518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.388518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.388835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad7d52-829b-4603-9c1c-a3315888ce85 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.388915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 291/576] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/role_assignments?user.id=ededf6d88dee4745978c8566940b2180&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:47.389538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.412014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.426461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.428007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.428299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.428679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.442990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.445080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:47.465817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:47.466805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:47.467697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:47.467697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:47.467697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:47.467697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:47.472167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 286/577] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:47.473166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.511351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dcbd6-cf0d-40d5-b1cb-1c3064696470 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.549594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.550273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.550550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.550828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.551581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:47.551922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.553325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.567844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 292/578] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:47.567844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.593384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee69382-5d00-4bcd-9be9-2a860723e84e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.621820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.630776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.631589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.631933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.632215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.641365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.642824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:47.659552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:47.660571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:47.661515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 287/579] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:47.662108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.668203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.688224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e95971-401c-4952-bafa-e4cb94c94fb8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.740519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.741821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.743098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.743533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.757177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.758849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.778069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2302ce12-9f87-429d-8e7d-2581eb1fed99 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:47.780056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:47.780923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:47.781822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:47.781822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:47.781822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:47.781822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:47.781822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 293/580] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:47.782105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.822314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.823485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.823803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.824083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.824812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:47.825122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.826509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.844908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 288/581] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:47.844908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.898391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.899112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e90f2-5c77-4159-8109-9b6a3aea0a16 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.935572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:47.940374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.941253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.941531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.941800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.953389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.955545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:47.957405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222b2aad-f9d9-453f-8233-9584217ea4a5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:47.972864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:47.973833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:47.974761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:47.974761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:47.974761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:47.974761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:47.982171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 294/582] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:47.982171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:47.995289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:47.996008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:47.996301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:47.996577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:47.997421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:47.997735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:47.999173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.018027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 289/583] 10.222.0.42 () {56 vars in 1137 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:48.019298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.099559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb314a59-545a-41fc-a033-e90c140cae7b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.125866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d2f9d4-8717-44ca-9113-e62fa94c936a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.134259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:48.138226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.138944 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.139237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.139448 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.140208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:48.140633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.141937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.154777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 295/584] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:48.155617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.168802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.169665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.169956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.170235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.177988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.179447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:48.193635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:48.194558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 290/585] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:48.196394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.216775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:48.277892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752147ae-3489-4589-9b23-6b00f2bd85b0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.304121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b95f36-b6c7-42f4-a6f5-89d1a464ed6c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.318937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.320012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.320295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.320988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.322018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:48.322334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.323765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.336611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 296/586] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:48.337359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.342565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.343361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.343776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.344642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.355458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.356864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:48.375814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:48.376728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:48.377636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:48.377636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:48.377636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:48.377636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:48.381469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 291/587] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:48.382382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.403583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:48.450892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea692aca-fe09-4453-a81a-d860225d7c5f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.454437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:48.486949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.487843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.488128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.488456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.488569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e81bf-8f5c-43d1-b4c8-3817a21ceaa6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.498243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.499734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:48.515122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:48.516164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:48.517106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 297/588] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:48.517690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.537364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.538345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.539121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.539461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.540287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:48.540603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.543112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.559347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 292/589] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:48.560233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.646012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:48.646525 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d775b00-71d3-4c54-a95c-cab38eed5f88 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.671626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235501c-8d39-4349-81f1-e1f488ada944 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.685562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.686334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.686576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.686864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.698768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.700846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.713132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.714553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.715025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.715307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.716067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:48.716370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.717741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:48.720521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:48.721467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:07:48.722414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:48.726470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 298/590] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:48.727316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.735683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 293/591] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:48.735683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.759381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:48.843267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bcc6e-7287-4fe0-af81-be2ea88b9fe0 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.846846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1665e718-cdd3-4594-a647-fe0c3b144cab None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:48.878855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.879771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.880061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.880340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/e757ff94b7474fd99a1c7221be19c64d/roles/7c5539dafdbe475babeae5e546b4038c` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.881734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:48.882405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:48.882704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:48.882963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:48.883838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:48.884149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.886189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.889707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:48.896319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=e757ff94b7474fd99a1c7221be19c64d, role_id=7c5539dafdbe475babeae5e546b4038c)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:48.897759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:48.906784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 299/592] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:48.907615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:48.984982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 294/593] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:48 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/e757ff94b7474fd99a1c7221be19c64d/roles/7c5539dafdbe475babeae5e546b4038c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:48.986091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.001676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:49.024996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e817c66-be83-40dd-97c0-c77ab7a5b3c1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.067308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.068050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.068338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.068684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.069561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:49.069882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.071858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.095010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 300/594] 10.222.0.42 () {56 vars in 1137 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:49.100066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.144411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ce3c-1905-4f6e-9ca7-b6e933602846 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.198269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.200216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.201313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.201984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.212832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.215174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.219050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb5ff1-7592-473e-b0a4-6d10dae78578 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:49.232385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:49.233341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:49.234229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:49.235203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 295/595] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:49.235681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.269717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.270615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.270864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.271151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.280880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:49.283103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.284694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.296117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:49.303722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:49.304809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:49.305809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:49.305809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:49.305809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:49.305809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:49.305809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 301/596] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:49.307953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.362322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117fac8-2c0d-40e8-b088-4792ec55e213 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.400715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.402543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.403056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.403532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.414450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a8d9a-aef5-4610-8434-f1d8bf3d3f2b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.417001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.419121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:49.441084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:49.442042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:49.443015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:49.443015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:49.443015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:49.443015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:49.445795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 296/597] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:49.447257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.458902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.459501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.463793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.463793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.463793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:49.463793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.463793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.480025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 302/598] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:49.480025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.532231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:49.572185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd94df-8615-4b14-9b7f-ae14092d7a5d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.576204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:49.583202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075fefb0-2e09-4fc0-a697-c702cf328e9d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.613754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.616534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.617946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.618400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.619522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:49.620561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.621688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6007b415-604f-418c-919c-e420d272096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.622432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007b415-604f-418c-919c-e420d272096a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.622602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.622778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007b415-604f-418c-919c-e420d272096a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.623176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6007b415-604f-418c-919c-e420d272096a None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.624107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007b415-604f-418c-919c-e420d272096a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:49.624497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007b415-604f-418c-919c-e420d272096a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.625875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6007b415-604f-418c-919c-e420d272096a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.636618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 297/599] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:49.639779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.642877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 303/600] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:49.644420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5a045-4c7e-43b7-946a-4c41eaf0e3cf None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.644815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5a045-4c7e-43b7-946a-4c41eaf0e3cf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.645196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5a045-4c7e-43b7-946a-4c41eaf0e3cf None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.647257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 304/601] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:49.648431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5a045-4c7e-43b7-946a-4c41eaf0e3cf None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.754448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12defb5-afeb-453b-8dd0-4abd8be49b1b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.757786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5a045-4c7e-43b7-946a-4c41eaf0e3cf None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.772799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:49.793065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7c739-560b-4b2d-af43-13875a510a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.794095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7c739-560b-4b2d-af43-13875a510a13 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.794379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7c739-560b-4b2d-af43-13875a510a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.794791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7c739-560b-4b2d-af43-13875a510a13 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/a98415ce308d41f6b8d5db6ca3f2ff19/roles/7c5539dafdbe475babeae5e546b4038c` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.795288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.796115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.796398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.796673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.803429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.804927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.812487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7c739-560b-4b2d-af43-13875a510a13 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=a98415ce308d41f6b8d5db6ca3f2ff19, role_id=7c5539dafdbe475babeae5e546b4038c)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.813980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7c739-560b-4b2d-af43-13875a510a13 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:49.819650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:49.821083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:49.822181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 305/602] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:49.826991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:49.921718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 298/603] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:49 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/a98415ce308d41f6b8d5db6ca3f2ff19/roles/7c5539dafdbe475babeae5e546b4038c => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:49.923351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05fb4-87ab-4452-b211-927f850658ef None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.923709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05fb4-87ab-4452-b211-927f850658ef None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.924090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd05fb4-87ab-4452-b211-927f850658ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.944217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b209beb-2539-424b-9c43-a2e2bd415eb4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:49.949777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bbd05fb4-87ab-4452-b211-927f850658ef None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:49.951127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bbd05fb4-87ab-4452-b211-927f850658ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:49.954736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:49.986608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:49.987513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:49.987904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:49.988257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:49.996683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:49.998305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:50.024943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:50.026006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 306/604] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:50.026923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:50.053853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:50.056918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 299/605] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:50.058992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05fb4-87ab-4452-b211-927f850658ef None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:50.144311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe52c-a96e-4715-b656-acb3285b598b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:50.196858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:50.198568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:50.199004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:50.199429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:50.199881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd05fb4-87ab-4452-b211-927f850658ef None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:50.200963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:50.201693 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:50.204313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:50.221168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 307/606] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:50.222339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:50.248058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:50.248733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:50.249017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:50.249300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:50.250095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:50.250407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:50.252606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:50.265923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:50.270873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 300/607] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:50.271695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:50.336840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:50.347042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bc998-d43d-4af1-9dc8-f76c88d7c44a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:50.386426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:50.387479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:50.387948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:50.388399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:50.399232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:50.407832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:50.409004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:50.409945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:50.410652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b820e-8003-4b0d-9704-c7cf54b802e0 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:50.417412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 308/608] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:50.418453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:50.452567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:50.453812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:50.454101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:50.454379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/eefd00457d354271867932d336d7f69c/roles/7c5539dafdbe475babeae5e546b4038c` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:50.475459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=eefd00457d354271867932d336d7f69c, role_id=7c5539dafdbe475babeae5e546b4038c)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:50.476956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:50.529828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfcb8e-f10f-4128-b09c-9a1a9f9e137a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:50.566488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:50.567376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:50.567656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:50.567958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/b6cf996e1fdb4a458d06bfd23d082dec/roles/7c5539dafdbe475babeae5e546b4038c` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:50.583331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=b6cf996e1fdb4a458d06bfd23d082dec, role_id=7c5539dafdbe475babeae5e546b4038c)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:50.584826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:50.611384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 301/609] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:50 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/eefd00457d354271867932d336d7f69c/roles/7c5539dafdbe475babeae5e546b4038c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:50.611384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:50.678027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 309/610] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:50 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/b6cf996e1fdb4a458d06bfd23d082dec/roles/7c5539dafdbe475babeae5e546b4038c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:50.683184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:50.750477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553733c-1769-4599-83a9-88ed3bc62ac4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:50.789770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:50.790463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:50.790747 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:50.791030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:50.792351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:50.792666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:50.794079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:50.804475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 302/611] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:50.810905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:50.820858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:50.927567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b64092-8818-4b0d-9932-c0bb1502d1df None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:50.968889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:50.969848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:50.970229 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:50.970604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:50.982888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:50.984533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:51.003116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:51.004238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:51.005331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:51.005331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:51.005331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:51.005331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:51.005331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 310/612] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:51.011481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:51.129003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e1ece-2b8d-4fc1-9712-0579d7777be8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:51.156534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:51.238592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8824b1-9921-4eff-96a0-ef7c7fe06e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:51.239201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8824b1-9921-4eff-96a0-ef7c7fe06e75 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:51.243777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8824b1-9921-4eff-96a0-ef7c7fe06e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:51.243777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8824b1-9921-4eff-96a0-ef7c7fe06e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:51.243777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8824b1-9921-4eff-96a0-ef7c7fe06e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:51.243777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8824b1-9921-4eff-96a0-ef7c7fe06e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:51.243777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8824b1-9921-4eff-96a0-ef7c7fe06e75 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:51.273475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 303/613] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:51.279715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:51.444092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:51.469927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ae93-0ece-4dac-aee0-68ca90ef5395 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:51.470295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ae93-0ece-4dac-aee0-68ca90ef5395 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:51.473559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496ae93-0ece-4dac-aee0-68ca90ef5395 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:51.480971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 304/614] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:07:51.492448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd09226-b5d9-47f4-982a-693de1dece9d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:51.492704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd09226-b5d9-47f4-982a-693de1dece9d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:51.492961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd09226-b5d9-47f4-982a-693de1dece9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:51.517678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72ab2f-db3d-4a9d-8459-231694f9b86e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:51.529310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-dfd09226-b5d9-47f4-982a-693de1dece9d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:51.530792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dfd09226-b5d9-47f4-982a-693de1dece9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:51.592315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:51.593875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:51.595197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:51.597134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] PATH_INFO: `/v3/users/swift` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:51.614728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:51.616996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:51.647167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:51.648217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 21:07:51.649172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:51.662005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 311/615] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:07:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:51.664081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:51.687844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 305/616] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:51.730972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd09226-b5d9-47f4-982a-693de1dece9d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:51.897049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be91a43b-a1dc-4476-be68-cd8dfcfb0fef None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:51.938544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:51.967554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:51.967923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:51.968330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:51.968468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:51.970293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:51.970293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:51.971079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.012789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd09226-b5d9-47f4-982a-693de1dece9d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.014927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:52.019082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 312/617] 10.222.0.42 () {56 vars in 1135 bytes} [Thu May 26 21:07:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:52.024450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.124501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.125330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.125672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.125989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.144165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db10d126-750f-45ed-8c95-4a2cb0aedb66 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.153085 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:52.167775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:52.168889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:52.169735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:52.172676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 306/618] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 442 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:52.178511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.184788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.185685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.185976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.186255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.197677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:52.199150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:52.218012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:52.218943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:52.219858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:52.219858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:52.219858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:52.219858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:52.219858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 313/619] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:07:52.227766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.251778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:52.330890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187ef77-bc73-467e-a882-5df4e0914d03 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.369975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.370651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.370937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.371219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.371976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:52.372287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:52.373682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.386803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 314/620] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:52.392202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.415731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a58e2-1178-473a-a065-6c642d5d9edb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.455300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e287b56-cc73-4195-a7e0-d5bbd69a3229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.456382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e287b56-cc73-4195-a7e0-d5bbd69a3229 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.456892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e287b56-cc73-4195-a7e0-d5bbd69a3229 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.457463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e287b56-cc73-4195-a7e0-d5bbd69a3229 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.458448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e287b56-cc73-4195-a7e0-d5bbd69a3229 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:52.458970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e287b56-cc73-4195-a7e0-d5bbd69a3229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:52.460576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e287b56-cc73-4195-a7e0-d5bbd69a3229 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.470344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 307/621] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:52.472036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b4d6c-fa0d-47c1-bbb4-4b899b1505c1 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.472533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b4d6c-fa0d-47c1-bbb4-4b899b1505c1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.473014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b4d6c-fa0d-47c1-bbb4-4b899b1505c1 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.478757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 308/622] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:52.480344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b903f9-3cad-4804-8df6-e5e9045ea8c9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.480842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b903f9-3cad-4804-8df6-e5e9045ea8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.481343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b903f9-3cad-4804-8df6-e5e9045ea8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.498756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-17b903f9-3cad-4804-8df6-e5e9045ea8c9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:52.500340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-17b903f9-3cad-4804-8df6-e5e9045ea8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:52.504243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:52.518464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f99f6-69aa-4f05-9da8-887a781946f3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.520357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:52.564098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.564971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.565696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.565983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.577885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:52.579454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.579971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 309/623] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:52.589355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b903f9-3cad-4804-8df6-e5e9045ea8c9 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:52.597294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:52.598545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:52.599421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 315/624] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:52.605041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.710556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b903f9-3cad-4804-8df6-e5e9045ea8c9 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.737670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34fea3-02a6-4bfc-80b5-78cab427a019 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.748706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.749806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.750272 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.750726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.761688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:52.763483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.781122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.781899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.782255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.782603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:52.782970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:52.783881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:52.784793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:52.784793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:52.784793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:52.784793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:52.785014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:52.785014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:52.786000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.787541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 310/625] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:52.788590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.800643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:52.817001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 316/626] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:52.817001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:52.818340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:52.913971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d075ce7-abdf-4f18-9daf-014804233495 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.943571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed8cc6-dbde-4545-a936-352d49e2b3d5 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:52.963952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.965087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.965553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.966043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.977400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:52.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:52.986486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:52.987315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:52.987965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:52.987965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:52.987965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:52.987965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:52.988316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:52.988316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:52.989939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:52.990766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 311/627] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:52.996856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.007835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 317/628] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:53.008983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.079061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.105402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.117857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee032966-8e72-47ae-badc-c6fe764af37c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.158486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833eeda-73a8-4102-ab2e-78c3f84a7c27 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.159313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.160586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.161197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.161680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/ededf6d88dee4745978c8566940b2180/roles/7c5539dafdbe475babeae5e546b4038c` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.178573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=ededf6d88dee4745978c8566940b2180, role_id=7c5539dafdbe475babeae5e546b4038c)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:53.180585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:53.199835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.200590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.200879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.201158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.201913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] RBAC: Adding query filter params (name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:53.202227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:53.203605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:53.215844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 318/629] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:53.215844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.262325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 312/630] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:07:52 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/ededf6d88dee4745978c8566940b2180/roles/7c5539dafdbe475babeae5e546b4038c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:07:53.263724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.391422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1cef-7d5c-44b7-baa8-746391c6abfc None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.407061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0f71a-4219-4c4f-a843-9b80787714f3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.429742 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.449998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.455621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e89b1-caf8-48e7-bf8b-b38a1b93ac1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.457066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e89b1-caf8-48e7-bf8b-b38a1b93ac1b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.457809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e89b1-caf8-48e7-bf8b-b38a1b93ac1b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.458098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e89b1-caf8-48e7-bf8b-b38a1b93ac1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.462861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0837af2-8abe-43c7-a39b-453df07c2b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.464565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0837af2-8abe-43c7-a39b-453df07c2b87 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.465075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0837af2-8abe-43c7-a39b-453df07c2b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.465552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0837af2-8abe-43c7-a39b-453df07c2b87 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.475476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e89b1-caf8-48e7-bf8b-b38a1b93ac1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:53.475476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e89b1-caf8-48e7-bf8b-b38a1b93ac1b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:53.475717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0837af2-8abe-43c7-a39b-453df07c2b87 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:53.477434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0837af2-8abe-43c7-a39b-453df07c2b87 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f0837af2-8abe-43c7-a39b-453df07c2b87 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:53.495578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:53.496638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 21:07:53.497595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-109e89b1-caf8-48e7-bf8b-b38a1b93ac1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:53.498140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:53.498924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:53.499872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:53.499872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:53.499872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:53.499872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:53.500105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 319/631] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:53.500175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 313/632] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:53.500242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62335db5-9cd5-412f-a40b-30894ef3d7d3 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.500242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62335db5-9cd5-412f-a40b-30894ef3d7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.500363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b4a0-9f0f-48ae-a612-0e2f16454f47 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.500439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62335db5-9cd5-412f-a40b-30894ef3d7d3 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.500513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b4a0-9f0f-48ae-a612-0e2f16454f47 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.500513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338b4a0-9f0f-48ae-a612-0e2f16454f47 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.505906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 314/633] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:53.505906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b4a0-9f0f-48ae-a612-0e2f16454f47 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.509653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 320/634] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:53.509653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4d513-dd59-40d8-983c-3236934456a2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.509653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4d513-dd59-40d8-983c-3236934456a2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.509653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a4d513-dd59-40d8-983c-3236934456a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.523930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-61a4d513-dd59-40d8-983c-3236934456a2 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:53.525160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-61a4d513-dd59-40d8-983c-3236934456a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:53.601476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 321/635] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:53.602782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cdf18-ed32-4aa5-b186-fba9ea61f160 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.603073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cdf18-ed32-4aa5-b186-fba9ea61f160 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.603346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cdf18-ed32-4aa5-b186-fba9ea61f160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.623346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ce2cdf18-ed32-4aa5-b186-fba9ea61f160 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:53.624555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ce2cdf18-ed32-4aa5-b186-fba9ea61f160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:53.638963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338b4a0-9f0f-48ae-a612-0e2f16454f47 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.675094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.675767 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.676091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.676362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.677206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:53.677515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:53.678954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:53.697130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 322/636] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:53.698943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cdf18-ed32-4aa5-b186-fba9ea61f160 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.711969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 315/637] 10.222.0.42 () {56 vars in 1139 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:53.712899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.741383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.757434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.815526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cdf18-ed32-4aa5-b186-fba9ea61f160 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.818967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1ed80-611d-4877-9d38-d0b813a0faa1 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.853201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.853414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:53.854389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.854669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.854765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:53.854940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.855044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:53.855273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:53.855660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:53.855987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:53.857377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:53.866650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:53.874343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 323/638] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:53.874343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:53.875312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:53.876322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:53.877171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 316/639] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:53.877878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:53.976678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.982836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed86f9b7-69c5-4de4-8663-dd98bfd7395f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:53.990339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:53.995990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06bd8ca-bbc3-427a-be76-f46c845bcc5f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.017637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.018291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.018571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.019007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.035280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.036106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.036388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.036660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.048487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:54.056308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:54.057845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:54.058680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 317/640] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:54.059403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.099410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.100901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.166008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c572ad8-cf3d-424b-ab01-a2f4af49b02d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.203049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.203896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.204182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.204462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.210294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 324/641] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:54.211444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.216024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.217476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.221318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:54.233342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:54.248661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 318/642] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.249627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:54.321561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe76532-dc80-418f-8d55-6bf4569f3c89 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.341199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7b23d-4ae0-4b70-bb8d-63d9e2fc9c48 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.357283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.357941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.358233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.358501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.359238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:54.359545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.360941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.369094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 325/643] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:54.370182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.380941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.381609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.381887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.382164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.382903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:54.383222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.385693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.397520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 319/644] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:54.397520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.442893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:54.466084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:54.481766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7cf24-cfbf-4c9b-a48a-e5f12e0e58f3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.502321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8164e-d6f4-4270-b326-afa03e821b62 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.518316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.519112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.519395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.519667 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.531111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.532602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.539350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.540050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.540334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.540605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.542033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:54.542348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.543775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:54.550642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:54.551716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:54.552851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:54.552851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:54.552851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:54.552851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:54.552851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 326/645] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:54.553136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.573963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 320/646] 10.222.0.42 () {56 vars in 1137 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:54.573963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.663443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed3f294-fbec-4d6f-805b-d74e89dffd39 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.684029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:54.687225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed0147f-0ad1-49f7-8f00-78fa23d5d53e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.704286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.705211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.705593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.705937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.718941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.720450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.728052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:54.729200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.730321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.730321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.730553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:54.739882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:54.740754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:54.741658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:54.741658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:54.741658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:54.741658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:54.741901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.741974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 327/647] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:54.741974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.742107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:54.770547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:54.771506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:54.772530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:54.772530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:54.772530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:54.772530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:54.773257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 321/648] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:54.774303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.870933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd8fe4-3041-4372-a428-f481a77b00c6 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.885655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddad740-4250-4b89-bee9-528c665eefd5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:54.905732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.906398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.906673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.906943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.907879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:54.908148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:54.909534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:54.922069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 328/649] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:54.922893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:54.928196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:54.929069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:54.929566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:54.929844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:54.958895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:54.983503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.006674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.008222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.038754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0314f-4695-4727-ad35-0f144856b904 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.076200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.076868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.077186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.077516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.078235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:55.078541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.079939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.092220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 329/650] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:55.093041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.123164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 322/651] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:55.124415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.194049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.203564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f5d7c1-e4ff-4e05-b3ae-a844dea67cd1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.210000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.235487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec28dca-45d4-42b3-80c7-681c25bbed96 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.238933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.239618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.239915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.240190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.240916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:55.241226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.242606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.254891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 330/652] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:55.256195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.272353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.273190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.273475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.273749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.282013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.283454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:55.298786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:55.299693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:55.300639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 323/653] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:55.301132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.383319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5bea9-df8d-4359-9902-e8716349e2a9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.412022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72590d1e-d1e2-4118-9a11-99a313c3da7e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.421178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.422033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.422299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.422577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.432099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.433532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.451344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:55.451735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:55.452676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:07:55.458489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:55.459044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.459044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.459044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.459260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 331/654] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:55.459260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.467119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.471489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.472986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.473126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:55.490838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:55.491946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:55.493081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:55.493081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:55.493081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:55.493081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:55.494649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 324/655] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:55.495575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.563486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9404b-0f94-4086-a6d6-fa5ee27e3254 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.602803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.603609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.604453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.604733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.609666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fe3670-5628-4186-b670-e173d1200b9e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.613013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.614656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:55.629706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:55.630860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 332/656] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:55.631865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.646224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.646959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.647312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.647656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.648548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:55.648859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.650286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.662976 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 325/657] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:55.663839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.706541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.720510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.734813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3febb67e-2401-45b9-ade9-9bd8db0bc213 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.767860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a20b0-2a81-401a-87c4-33b83170e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.768199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2db37d-ba75-4ac5-b022-8f75d214cd17 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.768653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a20b0-2a81-401a-87c4-33b83170e5fb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.768935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a20b0-2a81-401a-87c4-33b83170e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.769207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a20b0-2a81-401a-87c4-33b83170e5fb None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.770037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a20b0-2a81-401a-87c4-33b83170e5fb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:55.770343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a20b0-2a81-401a-87c4-33b83170e5fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.771797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a20b0-2a81-401a-87c4-33b83170e5fb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.790109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 333/658] 10.222.0.42 () {56 vars in 1137 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:55.791184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b3e2-38ce-47e7-a9b0-09989a35b7f2 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.792108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b3e2-38ce-47e7-a9b0-09989a35b7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.792381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926b3e2-38ce-47e7-a9b0-09989a35b7f2 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.793727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 334/659] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:55.794467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926b3e2-38ce-47e7-a9b0-09989a35b7f2 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.807946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.808608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.808890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.809163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.809882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:55.810290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.811905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.824005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 326/660] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:55.824572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.900126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926b3e2-38ce-47e7-a9b0-09989a35b7f2 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.923285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45522d2f-3b2d-4e8f-8a1b-1c6cb56fa7c6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:55.927977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.933940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:55.934187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.942782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.942782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.942782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.942782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:55.942782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.942782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.955251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 335/661] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:55.956092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:55.957504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d902c-4be7-499d-a56d-333bf804020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:55.958177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d902c-4be7-499d-a56d-333bf804020f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.958463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d902c-4be7-499d-a56d-333bf804020f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.958728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d902c-4be7-499d-a56d-333bf804020f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.959567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d902c-4be7-499d-a56d-333bf804020f None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=e757ff94b7474fd99a1c7221be19c64d) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:55.960192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d902c-4be7-499d-a56d-333bf804020f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:55.961643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d902c-4be7-499d-a56d-333bf804020f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:55.967643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 327/662] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/role_assignments?user.id=e757ff94b7474fd99a1c7221be19c64d&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:55.968875 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1a7fb-696e-4362-af48-511a114e3ce5 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:55.969164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1a7fb-696e-4362-af48-511a114e3ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:55.969442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1a7fb-696e-4362-af48-511a114e3ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:55.988512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-4fa1a7fb-696e-4362-af48-511a114e3ce5 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:55.989980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4fa1a7fb-696e-4362-af48-511a114e3ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:56.066574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 328/663] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:56.067449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1a7fb-696e-4362-af48-511a114e3ce5 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.078992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cefe52-cc5b-4106-bb3e-38dea3248833 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.119344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.120611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.120894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.121686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.187870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.192250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1a7fb-696e-4362-af48-511a114e3ce5 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.197343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.206783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:56.208331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:56.234902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.235986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.236279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.236556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.243695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:56.246137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:56.261522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:56.262472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:56.263369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 329/664] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:56.264382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.323341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 336/665] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:56.324216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.376671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11075f-248b-409c-85d9-0281762bbe00 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.413657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.414319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.414607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.414890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.423391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.437118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9098d2c-f045-48d7-9468-2790032f7851 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.448721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.479299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.480212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.480500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.480780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.491361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:56.492545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:07:56.492930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:07:56.500940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:56.501888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:07:56.502707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:56.506975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 337/666] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:56.507652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.610750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 330/667] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:56.610750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.617448 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681297d4-1672-4984-848a-070ed96a2c9e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.652198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.652790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.653067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.653339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.654133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:56.654444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:56.655919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:56.668765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 338/668] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:56.669593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.681510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.685632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.729973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce19f3-b033-4c59-8980-4b5e29ccae73 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.768492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.769299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.769584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.769862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.781606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:56.784148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:56.792604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856160-8c01-489f-ad4a-1027b374bd55 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:56.802947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:56.804068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:56.804957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:56.804957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:56.804957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:56.804957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:56.804957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 331/669] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:56.804957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.835485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.836187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.836493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.836948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.837489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:56.837800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:56.839723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:56.848108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 339/670] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:56.853157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:56.917999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed73ee5-1314-485c-91fd-6a73a66fbe8f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.934927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.935071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:56.953892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:56.954754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:56.955033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:56.955310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:56.967160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:56.967303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de674f3-973d-4019-ada8-726dab2c1d6e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:56.968772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:56.987101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:56.988154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:56.989054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:56.989054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:56.989054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:56.989054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:56.992068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 332/671] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:56.992969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.002685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.003716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.004249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.004728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.005913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=a98415ce308d41f6b8d5db6ca3f2ff19) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:57.006430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.008155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.017992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 340/672] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/role_assignments?user.id=a98415ce308d41f6b8d5db6ca3f2ff19&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:57.020069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.117429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3aab3-caf8-4639-8dbb-7c671e540d5b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.144699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71b7036-7be0-4ecc-b129-bc0f9411adc0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.158359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.159507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.159927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.160302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.161144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:57.161712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.163290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.177954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 333/673] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:57.179023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.196140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.198519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.198983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.199437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.202990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:57.214483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.215870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:57.217928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:57.242928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:57.243939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:57.245429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:57.245429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:57.245429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:57.245429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:57.253119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 341/674] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:57.254076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.308070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccab93-35c8-476f-af37-f3f9dc8d55d5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.352776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.353861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.354130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.354475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.355210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:57.355523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.356923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.371310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 334/675] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:57.375768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.383160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725d34f-ee26-45d0-880d-39f10a148f50 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.421375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.422175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.422460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.422734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.429881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.431298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:57.448757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:57.449665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:57.450551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 342/676] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:57.451037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.487812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:57.500138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e06d6-a539-4b6d-8347-8edd20df12f9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.516847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:57.537541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.538141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.538430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.538708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.539437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:57.539763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.541304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.554429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 335/677] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:57.559792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.589419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bebec-7125-4355-99f3-b8f6682db5f9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.684562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a445bcd-e82c-44c3-97e2-c9ca1c5c6888 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.728453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.730031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.730525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.731105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.732131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:57.732739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.734376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.758911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.759733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.760077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.760354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.768566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.770098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:57.785434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:57.786405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:57.787287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:57.791978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 343/678] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:57.792992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.885867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 336/679] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:57.887184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:57.905000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9881bf-9363-475e-b9f3-aa7597d7d6c2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:57.909077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:57.912694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:57.940817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:57.941695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:57.941985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:57.942261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:57.951541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:57.953483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:07:57.972879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:57.973876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 21:07:57.974819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:57.979595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 344/680] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:57.980843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.032033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efb315-c774-4b08-ba56-765d691004b7 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.077501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.078851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.079934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.080438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.081711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:58.082533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.084686 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.100941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e5b0d-b53e-45e7-8b79-004d98d49d5a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.102066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 337/681] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:58.103099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.146810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.147462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.147764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.148053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.148890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=b6cf996e1fdb4a458d06bfd23d082dec) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:58.149206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.150684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.163791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 345/682] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:57 2022] GET /identity/v3/role_assignments?user.id=b6cf996e1fdb4a458d06bfd23d082dec&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:58.164889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.189684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:58.228568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:58.235540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f04ccb-5de0-4749-95b3-e4bac57f3430 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.277543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb8b8d0-af2a-4418-8afc-91b79c46e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.278740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b8d0-af2a-4418-8afc-91b79c46e627 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.279229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b8d0-af2a-4418-8afc-91b79c46e627 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.279734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb8b8d0-af2a-4418-8afc-91b79c46e627 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.280964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8b8d0-af2a-4418-8afc-91b79c46e627 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:58.281591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8b8d0-af2a-4418-8afc-91b79c46e627 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.283257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb8b8d0-af2a-4418-8afc-91b79c46e627 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.305654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 338/683] 10.222.0.42 () {56 vars in 1135 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:58.307606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163af5-a1d1-4ed8-81f2-8ac56d3c86c3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.308182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163af5-a1d1-4ed8-81f2-8ac56d3c86c3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.308659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22163af5-a1d1-4ed8-81f2-8ac56d3c86c3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.310540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 339/684] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:58.312122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22163af5-a1d1-4ed8-81f2-8ac56d3c86c3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.345536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0199af77-5f75-486d-af53-5f51724c38b0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.396918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5120a8-01c5-4ee1-8bb4-ec0803f75163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.397626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5120a8-01c5-4ee1-8bb4-ec0803f75163 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.398173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5120a8-01c5-4ee1-8bb4-ec0803f75163 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.398456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5120a8-01c5-4ee1-8bb4-ec0803f75163 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.399307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5120a8-01c5-4ee1-8bb4-ec0803f75163 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=eefd00457d354271867932d336d7f69c) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:58.399623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5120a8-01c5-4ee1-8bb4-ec0803f75163 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.402969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5120a8-01c5-4ee1-8bb4-ec0803f75163 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.412362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 346/685] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/role_assignments?user.id=eefd00457d354271867932d336d7f69c&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 608 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:58.413504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c24ff-dc85-4d5e-a87e-3d666598234a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.413796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c24ff-dc85-4d5e-a87e-3d666598234a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.414079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c24ff-dc85-4d5e-a87e-3d666598234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.435010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-377c24ff-dc85-4d5e-a87e-3d666598234a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:58.436732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-377c24ff-dc85-4d5e-a87e-3d666598234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:58.442585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22163af5-a1d1-4ed8-81f2-8ac56d3c86c3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.487632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:58.498849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.500547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.501495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.502316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.545324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 347/686] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:07:58.568749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c24ff-dc85-4d5e-a87e-3d666598234a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.600536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:58.603357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.606393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.707041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c24ff-dc85-4d5e-a87e-3d666598234a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.739994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 340/687] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:07:58.744745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.753313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.754457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.755550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.756045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] PATH_INFO: `/v3/services/network` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.756919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.759447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:58.777823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:58.778916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:07:58.779774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:58.781431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 348/688] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/services/network => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:58.788002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:58.788879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.871775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0710b40e-80ed-48c7-a013-583b2eff3c54 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.892954 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:58.906730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d8311-f523-478f-86ab-459c594d5599 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:58.915959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.916873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.917155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.917426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.931122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.933153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.948380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:58.949411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:58.949906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:58.950411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:58.951357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] RBAC: Adding query filter params (name=network) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:58.951918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:58.953595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:07:58.957871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:58.958809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:58.959722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:07:58.959722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:07:58.959722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:07:58.959722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:07:58.961993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 341/689] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:58.964200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 349/690] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:58.965621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:58.970309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:59.067675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:59.095839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b384fc-ed02-40c0-b67d-02b772c9915c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:59.102811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dffd4f3-51c6-4a4e-96ce-edd702ce8754 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:59.160689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:59.161387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:59.161840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:59.162276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:59.163582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:59.164162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:59.167112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:59.172325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:59.173004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:59.173288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:59.173564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:59.174278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] RBAC: Adding query filter params (type=network) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:59.174585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:59.175981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:59.187117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 350/691] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:59.202566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 342/692] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:59.208610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:59.240163 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:59.372740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af848b9-d712-484d-a143-548cdb00a164 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:59.425696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4861a4d-e82f-4f79-ae88-98d33fe5168a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:59.426520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4861a4d-e82f-4f79-ae88-98d33fe5168a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:59.426808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4861a4d-e82f-4f79-ae88-98d33fe5168a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:59.427087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4861a4d-e82f-4f79-ae88-98d33fe5168a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:59.435371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4861a4d-e82f-4f79-ae88-98d33fe5168a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:59.436893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4861a4d-e82f-4f79-ae88-98d33fe5168a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-e4861a4d-e82f-4f79-ae88-98d33fe5168a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:07:59.456635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:07:59.457589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:07:59.458465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 351/693] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:07:59.465826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:59.574298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86b9f85-7b7a-489c-8b5b-dfd7311ff69d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:59.599614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2744e-63cb-4125-bd79-6e95d067336b None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:59.599991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2744e-63cb-4125-bd79-6e95d067336b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:59.600218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd2744e-63cb-4125-bd79-6e95d067336b None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:59.601826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 352/694] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:07:59.607412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d673c-97dc-48ed-a0e0-3226d664ff87 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:59.607699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d673c-97dc-48ed-a0e0-3226d664ff87 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:59.608027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7d673c-97dc-48ed-a0e0-3226d664ff87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:59.614255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:59.615213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:59.615498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:59.615806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:59.616603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:59.616916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:59.618341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:59.626933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0f7d673c-97dc-48ed-a0e0-3226d664ff87 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:07:59.629273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0f7d673c-97dc-48ed-a0e0-3226d664ff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:07:59.630905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 343/695] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:07:59.631173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:59.637073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:59.754183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 353/696] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:07:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:07:59.783481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d673c-97dc-48ed-a0e0-3226d664ff87 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:07:59.819275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:07:59.824434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8234c5-ee1c-474f-b631-e57356136b6e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:07:59.902273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:07:59.902909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:07:59.903195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:07:59.903471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:07:59.904342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=ededf6d88dee4745978c8566940b2180) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:07:59.904659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:07:59.906094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:07:59.921181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 344/697] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:07:59 2022] GET /identity/v3/role_assignments?user.id=ededf6d88dee4745978c8566940b2180&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:07:59.970025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7d673c-97dc-48ed-a0e0-3226d664ff87 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:00.008892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:00.009709 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:00.010104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:00.010385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:00.022369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:00.023955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:00.043909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:00.044862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:00.045786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:00.045786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:00.045786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:00.045786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:00.045786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 354/698] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:00.052396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:00.149640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:00.212103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f87f76-b9bd-4215-aa8e-5cc9ea2e102d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:00.249778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d52022-49ed-4ded-8fb6-2481132df735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:00.250814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d52022-49ed-4ded-8fb6-2481132df735 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:00.251331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d52022-49ed-4ded-8fb6-2481132df735 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:00.251834 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d52022-49ed-4ded-8fb6-2481132df735 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:00.252761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d52022-49ed-4ded-8fb6-2481132df735 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:00.253280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d52022-49ed-4ded-8fb6-2481132df735 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:00.254828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d52022-49ed-4ded-8fb6-2481132df735 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:00.269332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 345/699] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:00.275028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:00.337624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:00.382074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44ac274-a093-4e72-a5eb-0beaaeb95b2c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:00.417957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907b806-3c08-4526-8f39-32168885f22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:00.418969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907b806-3c08-4526-8f39-32168885f22e None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:00.419450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907b806-3c08-4526-8f39-32168885f22e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:00.419950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907b806-3c08-4526-8f39-32168885f22e None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:00.420913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907b806-3c08-4526-8f39-32168885f22e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:00.422614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907b806-3c08-4526-8f39-32168885f22e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:00.511520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 355/700] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:08:00 2022] POST /identity/v3/users => generated 254 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:01.153485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d999538-85ba-4c1e-b3f3-44d98681b6f7 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.154037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d999538-85ba-4c1e-b3f3-44d98681b6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.154037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d999538-85ba-4c1e-b3f3-44d98681b6f7 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.155679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 346/701] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:01.161038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3293ee-05ab-476a-9190-f8415a51a682 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.161477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3293ee-05ab-476a-9190-f8415a51a682 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.161653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3293ee-05ab-476a-9190-f8415a51a682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.180125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-cc3293ee-05ab-476a-9190-f8415a51a682 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:01.181322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cc3293ee-05ab-476a-9190-f8415a51a682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:01.266931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 356/702] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:08:01.309194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d999538-85ba-4c1e-b3f3-44d98681b6f7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:01.448228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c06e2-b185-47ec-be0e-538af06c7f8e None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.448903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c06e2-b185-47ec-be0e-538af06c7f8e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.449390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c06e2-b185-47ec-be0e-538af06c7f8e None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.451242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 357/703] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:01.456740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb2e3d-7eca-4dae-bc5e-4ae0d744c942 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.457329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb2e3d-7eca-4dae-bc5e-4ae0d744c942 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.457838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb2e3d-7eca-4dae-bc5e-4ae0d744c942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.459621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d999538-85ba-4c1e-b3f3-44d98681b6f7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:01.484514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-48bb2e3d-7eca-4dae-bc5e-4ae0d744c942 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:01.486112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-48bb2e3d-7eca-4dae-bc5e-4ae0d744c942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:01.500083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:01.521279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:01.522200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.522568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.522850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.523611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:01.525124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:01.533302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:01.546040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:01.547820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:01.548905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:01.551157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 347/704] 10.222.0.42 () {56 vars in 1129 bytes} [Thu May 26 21:08:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:01.556954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:01.568577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 358/705] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:08:01.594795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb2e3d-7eca-4dae-bc5e-4ae0d744c942 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:01.690137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2286b-5e29-4f1a-95d6-3c408abbfc9e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:01.728949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bb2e3d-7eca-4dae-bc5e-4ae0d744c942 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:01.740477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:01.741158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.741631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.742072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.742937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:01.743422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:01.745498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:01.756190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 348/706] 10.222.0.42 () {56 vars in 1124 bytes} [Thu May 26 21:08:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:01.761991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:01.768280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:01.769412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.769914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.770395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.782653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:01.791233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:01.792917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:01.793963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:01.796278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 359/707] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:01.801933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:01.840185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:01.854719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:01.914775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac3525-74c3-4998-b71f-cd461e9267c4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:01.917172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c75ee3-ce6b-4625-bcd3-b8f774dd3754 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:01.953135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e95563c-2cf6-4f3b-9e51-87be176087bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:01.953572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:01.954253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.954422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95563c-2cf6-4f3b-9e51-87be176087bf None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.954557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.954802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.955182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95563c-2cf6-4f3b-9e51-87be176087bf None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.955613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:01.955726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e95563c-2cf6-4f3b-9e51-87be176087bf None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.955957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:01.956798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95563c-2cf6-4f3b-9e51-87be176087bf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:01.957275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95563c-2cf6-4f3b-9e51-87be176087bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:01.957387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:01.958860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e95563c-2cf6-4f3b-9e51-87be176087bf None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:01.966354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 349/708] 10.222.0.42 () {56 vars in 1124 bytes} [Thu May 26 21:08:01 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:01.967473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 360/709] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:01.968333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb68da1-b080-46a6-ad0a-53b08ee4c27c None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.968856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb68da1-b080-46a6-ad0a-53b08ee4c27c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.969366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb68da1-b080-46a6-ad0a-53b08ee4c27c None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:01.971226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 350/710] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:01.975436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:01.977066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13e201-2737-4615-8b29-743bbd076c07 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:01.977519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13e201-2737-4615-8b29-743bbd076c07 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:01.977917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13e201-2737-4615-8b29-743bbd076c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:02.012838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-aa13e201-2737-4615-8b29-743bbd076c07 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:02.014313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-aa13e201-2737-4615-8b29-743bbd076c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:02.087855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:02.096236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d77e4-51a7-498f-91e0-7e81ec5f5dfb None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:02.137406 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:02.138271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:02.138545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:02.138818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:02.168777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:02.170290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:02.178248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 351/711] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:02.196009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:02.196941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:02.199405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:02.199405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:02.199405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:02.199405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:02.199405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 361/712] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:02.203581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13e201-2737-4615-8b29-743bbd076c07 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:02.218098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:02.265808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:02.324647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13e201-2737-4615-8b29-743bbd076c07 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:02.335553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eda475-5451-4466-8e8c-ad7a9bc4298f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:02.361248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:02.361920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:02.362205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:02.362491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:02.363221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:02.363531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:02.364940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:02.372083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:02.372755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:02.373093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:02.373375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:02.374102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:02.375482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:02.378450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 352/713] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:02.386898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:02.505650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d54a3b-d93b-4796-a744-4dc74a175f4d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:02.508110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:02.533610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 362/714] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:02 2022] POST /identity/v3/services => generated 240 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:02.566719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:02.567593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:02.567923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:02.568199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:02.578759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:02.580397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:02.596599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:02.597647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:02.598591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 353/715] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:08:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:02.604000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:02.616700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:02.733103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24675-d0e8-4844-921e-04ac91bfd41d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:02.787570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a66ff6-126b-4d34-a2f6-c9e188f5f1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:02.788195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a66ff6-126b-4d34-a2f6-c9e188f5f1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:02.788482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a66ff6-126b-4d34-a2f6-c9e188f5f1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:02.788760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a66ff6-126b-4d34-a2f6-c9e188f5f1f9 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:02.789615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a66ff6-126b-4d34-a2f6-c9e188f5f1f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:02.789928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a66ff6-126b-4d34-a2f6-c9e188f5f1f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:02.792031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a66ff6-126b-4d34-a2f6-c9e188f5f1f9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:02.826012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 363/716] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:08:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:02.832927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:02.954665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c6c3b7-4117-4078-9046-56c56a43e7b5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:02.999376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:03.000158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.000450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.000729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.029602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:03.071777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:03.073184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:03.095896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557befbe-81b2-49ac-ad31-f979d8fb715c None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.096468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557befbe-81b2-49ac-ad31-f979d8fb715c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.096946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557befbe-81b2-49ac-ad31-f979d8fb715c None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.102543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 364/717] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:03.105038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd23b-1f06-4c9e-9225-eb27b20e250f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.105510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd23b-1f06-4c9e-9225-eb27b20e250f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.105980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebd23b-1f06-4c9e-9225-eb27b20e250f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.138022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0eebd23b-1f06-4c9e-9225-eb27b20e250f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:03.139594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0eebd23b-1f06-4c9e-9225-eb27b20e250f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:03.236865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:03.250421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 365/718] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:03 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 21:08:03.252998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 354/719] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:02 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 21:08:03.258881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd23b-1f06-4c9e-9225-eb27b20e250f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:03.274969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:03.350763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:03.424165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca4e41-638d-411e-b3f6-68bb68fb268d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:03.453906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebd23b-1f06-4c9e-9225-eb27b20e250f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:03.470762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:03.471948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.472467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.472947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.491731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:03.492901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.493403 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.493881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.504741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:03.506796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:03.508879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:03.520728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:03.521671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:03.527991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 355/720] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:03.538503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:03.539380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:03.540311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:03.540311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:03.540311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:03.540311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:03.543403 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 366/721] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:03.548695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:03.548902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:03.605792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:03.657634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6ddf6-66f0-4126-b42b-ad8185044a4e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:03.670414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437380ec-e559-461d-8c1d-c1a1929960b2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:03.693091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a00499-bd49-4630-98c1-2af45199c7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:03.693772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00499-bd49-4630-98c1-2af45199c7eb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.694053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00499-bd49-4630-98c1-2af45199c7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.694328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a00499-bd49-4630-98c1-2af45199c7eb None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.695059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00499-bd49-4630-98c1-2af45199c7eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:03.695627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00499-bd49-4630-98c1-2af45199c7eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:03.697048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a00499-bd49-4630-98c1-2af45199c7eb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:03.705279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:03.709277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d75c4c8-2bff-47b4-856a-b1f38e1cb61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:03.710240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75c4c8-2bff-47b4-856a-b1f38e1cb61e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.710351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 367/722] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:03.710823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75c4c8-2bff-47b4-856a-b1f38e1cb61e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.711300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d75c4c8-2bff-47b4-856a-b1f38e1cb61e None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.715797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c160-1745-4347-bc95-082c5a0f76f8 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.715797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c160-1745-4347-bc95-082c5a0f76f8 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.715797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c160-1745-4347-bc95-082c5a0f76f8 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.715797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 368/723] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:03.717353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75c4c8-2bff-47b4-856a-b1f38e1cb61e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:03.717822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75c4c8-2bff-47b4-856a-b1f38e1cb61e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:03.719348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d75c4c8-2bff-47b4-856a-b1f38e1cb61e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:03.719464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c160-1745-4347-bc95-082c5a0f76f8 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:03.729283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 356/724] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:03.730835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788ce1-e730-454d-abd7-1f95560f5671 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.731369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788ce1-e730-454d-abd7-1f95560f5671 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.731871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1788ce1-e730-454d-abd7-1f95560f5671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.753484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f1788ce1-e730-454d-abd7-1f95560f5671 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:03.755010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f1788ce1-e730-454d-abd7-1f95560f5671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:03.829838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 357/725] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:03 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:08:03.831055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788ce1-e730-454d-abd7-1f95560f5671 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:03.873510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c160-1745-4347-bc95-082c5a0f76f8 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:03.883209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:03.910702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:03.912447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:03.912947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:03.913452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:03.924143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:03.926277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:03.943358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:03.944480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:03.945685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:03.949729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 369/726] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:03.950669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:03.960326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1788ce1-e730-454d-abd7-1f95560f5671 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:03.979698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:04.010704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.011920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.012495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.013268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.029894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.031573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:04.055628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:04.057421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:04.058514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:04.058514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:04.058514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:04.058514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:04.061557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 358/727] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:04.062629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.076006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c322c-0e2e-4bd8-9c1e-ea15e3e60643 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.120405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.121550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.122021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.122477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.133807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:04.142199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:04.143231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:08:04.144137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:04.147040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 370/728] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:08:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:04.148083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.186405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:04.192139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f50cd6-a2c8-4829-976e-e1a650e9bada None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.233672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.234336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.234619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.234895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.235863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:04.236177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.237744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.259616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 359/729] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:04.261117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.280367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862883b-e271-40e1-99ef-25d8d89b2594 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.288504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:04.327425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.328697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.329399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.329831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.330866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:04.331728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.333676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.351611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 371/730] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:04.352683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.378321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef72e57-8b2a-4606-bdfb-6388c3d739d8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.415630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.416813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.417108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.417374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.418220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:04.418532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.420198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.434213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 360/731] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:04.435227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.462535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:04.475644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8befc1aa-89e3-4f29-8993-07678f9dd288 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.515338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.516331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.516616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.516891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.518296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=eefd00457d354271867932d336d7f69c) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:04.519050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.520601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.529004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 372/732] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/role_assignments?user.id=eefd00457d354271867932d336d7f69c&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:04.532753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.572538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:04.601914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6783e-3c5c-4849-8d8a-43b1ec31e4a1 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.670603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acccbc2-1286-4b13-a3f8-e34f2a63b4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.671756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acccbc2-1286-4b13-a3f8-e34f2a63b4b7 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.672189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acccbc2-1286-4b13-a3f8-e34f2a63b4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.672674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acccbc2-1286-4b13-a3f8-e34f2a63b4b7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.680904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8015bd-8f43-42ce-b679-f49c4240aad2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.685005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acccbc2-1286-4b13-a3f8-e34f2a63b4b7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.691379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acccbc2-1286-4b13-a3f8-e34f2a63b4b7 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-8acccbc2-1286-4b13-a3f8-e34f2a63b4b7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:04.717426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:04.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 21:08:04.719949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:04.722454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 361/733] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:04.723918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b14d1d-fc1c-462d-a1fb-ed5b256b7721 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.727655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b14d1d-fc1c-462d-a1fb-ed5b256b7721 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.728273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b14d1d-fc1c-462d-a1fb-ed5b256b7721 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.729590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 362/734] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:04.731328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b14d1d-fc1c-462d-a1fb-ed5b256b7721 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.737734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86652bfb-d250-444b-a398-da0c7a566435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.738821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86652bfb-d250-444b-a398-da0c7a566435 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.739261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86652bfb-d250-444b-a398-da0c7a566435 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.739716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86652bfb-d250-444b-a398-da0c7a566435 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.751818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86652bfb-d250-444b-a398-da0c7a566435 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.753554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86652bfb-d250-444b-a398-da0c7a566435 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-86652bfb-d250-444b-a398-da0c7a566435 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:04.773773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:04.774911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:04.776508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:04.776508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:04.776508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:04.776508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:04.777941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 373/735] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:04.782727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285f33d-5572-4256-bdbc-4d097e01a385 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.782727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285f33d-5572-4256-bdbc-4d097e01a385 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.782727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1285f33d-5572-4256-bdbc-4d097e01a385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.788742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:04.801659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1285f33d-5572-4256-bdbc-4d097e01a385 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:04.803204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1285f33d-5572-4256-bdbc-4d097e01a385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:04.870998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b14d1d-fc1c-462d-a1fb-ed5b256b7721 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:04.900151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 374/736] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:04 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 21:08:04.902030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285f33d-5572-4256-bdbc-4d097e01a385 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.927157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:04.928753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:04.929585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:04.930102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:04.931505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:04.932117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:04.934280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:04.972852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 363/737] 10.222.0.42 () {56 vars in 1135 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:04.973791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:04.977568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:05.035377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1285f33d-5572-4256-bdbc-4d097e01a385 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:05.078230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:05.079110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.079483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.079973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.080846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:05.081280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:05.082905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:05.084700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:05.090225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a3fc8-a295-4178-a826-661f7e9bed89 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:05.102805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 375/738] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:05.104089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:05.130114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:05.131157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.131589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.132086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.133057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:05.134585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:05.235874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98927a0-d990-4564-8fbb-d5d9cce47fbb None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:05.236551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 364/739] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:04 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:05.237744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:05.283872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73103f2a-a09b-42d0-8874-b6c833924691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:05.284972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73103f2a-a09b-42d0-8874-b6c833924691 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.285261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73103f2a-a09b-42d0-8874-b6c833924691 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.285535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73103f2a-a09b-42d0-8874-b6c833924691 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.331684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:05.375615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640e800-b779-4ebf-bdd1-df80046c3b7c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:05.384204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:05.398440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73103f2a-a09b-42d0-8874-b6c833924691 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:05.400323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73103f2a-a09b-42d0-8874-b6c833924691 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:05.419381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f915b7-9339-4937-92bc-09b8416b4845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:05.420429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f915b7-9339-4937-92bc-09b8416b4845 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.420845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f915b7-9339-4937-92bc-09b8416b4845 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.421371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f915b7-9339-4937-92bc-09b8416b4845 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.430579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f915b7-9339-4937-92bc-09b8416b4845 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:05.432903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f915b7-9339-4937-92bc-09b8416b4845 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-86f915b7-9339-4937-92bc-09b8416b4845 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:05.452771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:05.453769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:05.454707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 365/740] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:08:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:05.456902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c19c1-808e-457d-ba39-39f24ed99c44 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.456902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c19c1-808e-457d-ba39-39f24ed99c44 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.456902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c19c1-808e-457d-ba39-39f24ed99c44 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.461720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 366/741] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:05.462591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c19c1-808e-457d-ba39-39f24ed99c44 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:05.544207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 376/742] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:05.551139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6ce56-5ac7-43cf-a049-29b6594517c3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.551687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6ce56-5ac7-43cf-a049-29b6594517c3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.552723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae6ce56-5ac7-43cf-a049-29b6594517c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.575517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3ae6ce56-5ac7-43cf-a049-29b6594517c3 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:05.577613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3ae6ce56-5ac7-43cf-a049-29b6594517c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:05.590477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c19c1-808e-457d-ba39-39f24ed99c44 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:05.617382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:05.629486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:05.630229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.630704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.631059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.632060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:05.632492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:05.634197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:05.654942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 367/743] 10.222.0.42 () {56 vars in 1149 bytes} [Thu May 26 21:08:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:05.660259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:05.669462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 377/744] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:05.671217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6ce56-5ac7-43cf-a049-29b6594517c3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:05.700003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:05.784059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424078f4-f1ba-4511-babb-c4d21ccc4ae8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:05.805993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae6ce56-5ac7-43cf-a049-29b6594517c3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:05.837258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:05.839578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.840199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.840808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.844708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:05.845427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:05.845754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:05.845981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:05.853457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:05.855478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:05.880583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:05.881534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:05.882434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:05.882434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:05.882434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:05.882434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:05.883904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 368/745] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:05.885323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:05.922885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:05.924476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:05.935770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:05.948476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.008828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c39187-3d7f-4c6c-942e-09dba39ab8ec None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.046145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.046943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.047225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.047535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] PATH_INFO: `/v3/services/network` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.048196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.049555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.051261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 378/746] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:06.052402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:06.066830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:06.067859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:06.068809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 369/747] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:05 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:06.069703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.182989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.185085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f1785-d620-4f56-9e79-9fc232ac5d75 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.188969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1c127-4af8-4e8d-84d9-0f9332c15aaa None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.206982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.222761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.223586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.224015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.224292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.225064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:06.225756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.227799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.235985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.236747 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.237042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.239025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.248219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 379/748] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:06.248219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.255357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.257016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:06.276237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:06.277183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:06.278090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:06.278090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:06.278090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:06.278090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:06.282078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 370/749] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:06.282932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.363509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edf7fc-8197-47c7-9295-ed358a2f9c06 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.386221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd513ed-779d-4ae6-9018-38bbaaa13dbc None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.402590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.403778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.404290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.404775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.405686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] RBAC: Adding query filter params (name=network) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:06.406219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.407923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.413762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 380/750] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:06.414859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.421252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.422054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.422334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.422716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.429954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.431449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:06.446435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:06.447980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:06.448890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:06.453014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 371/751] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:06.453874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.466778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.471626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.530708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b186b-ce15-4ce8-9f52-8816ace19120 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.564035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3789f66-aa70-402b-ae84-d2136a84655b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.569224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.570419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.570947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.571436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.572479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:06.573024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.574783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.588802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 381/752] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:06.590040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.605620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.606430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.606665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.606946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.607663 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:06.608038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.609519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.615471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 372/753] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:06.616027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.721000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.749878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5af06-0222-431f-9386-915644835ef7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.751899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406f7bfe-2fef-4d48-ac7c-601db437a5d0 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.753499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.788379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.789456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.789948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.790426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.791440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:06.792039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.792511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.793305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.793587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.793860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.794629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.801003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.803419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.808213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 382/754] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:06.809288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:06.824369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:06.825308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:06.826218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:06.830529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 373/755] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:06.831373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.918290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d315da-6276-485b-9579-fc2c850d2a38 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.938233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790f8c6-5fe0-46f6-b0db-e0f47569e72b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:06.961511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.962657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.963200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.963675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.964848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] RBAC: Adding query filter params (interface=public, service_id=7381a391357347bab85803c7ee7cd230, region_id=RegionOne) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:06.965215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.966612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.974813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 383/756] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/endpoints?service_id=7381a391357347bab85803c7ee7cd230&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:06.975873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:06.978896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dce653-d7ad-4653-94bb-591aa7cfd1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:06.979865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce653-d7ad-4653-94bb-591aa7cfd1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.980159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce653-d7ad-4653-94bb-591aa7cfd1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.981212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dce653-d7ad-4653-94bb-591aa7cfd1d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.981326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce653-d7ad-4653-94bb-591aa7cfd1d8 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=ededf6d88dee4745978c8566940b2180) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:06.981581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce653-d7ad-4653-94bb-591aa7cfd1d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:06.983020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dce653-d7ad-4653-94bb-591aa7cfd1d8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:06.985281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:06.990123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 374/757] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/role_assignments?user.id=ededf6d88dee4745978c8566940b2180&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:06.991253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce675025-e433-455f-88a2-759aa8146f20 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:06.991595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce675025-e433-455f-88a2-759aa8146f20 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:06.991930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce675025-e433-455f-88a2-759aa8146f20 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:06.998178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 375/758] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:06.998178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce675025-e433-455f-88a2-759aa8146f20 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:07.031686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:07.128193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce675025-e433-455f-88a2-759aa8146f20 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:07.128402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486d57f-953b-45de-86e4-263741fa6d8a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:07.170472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8b2ba-4360-48dd-82d5-6df65df183f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:07.171548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8b2ba-4360-48dd-82d5-6df65df183f9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.171664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8b2ba-4360-48dd-82d5-6df65df183f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.172009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8b2ba-4360-48dd-82d5-6df65df183f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.172921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8b2ba-4360-48dd-82d5-6df65df183f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:07.173292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8b2ba-4360-48dd-82d5-6df65df183f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:07.174939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8b2ba-4360-48dd-82d5-6df65df183f9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:07.182829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:07.183137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.183321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.184178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.184383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:07.185676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:07.190528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 384/759] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:07.191879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed286407-a1d7-48e2-9d9f-c436b6284bc7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.192171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed286407-a1d7-48e2-9d9f-c436b6284bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.192468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed286407-a1d7-48e2-9d9f-c436b6284bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.193869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 376/760] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:06 2022] GET /identity/v3/services => generated 772 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:07.198159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:07.227374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ed286407-a1d7-48e2-9d9f-c436b6284bc7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:07.231276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ed286407-a1d7-48e2-9d9f-c436b6284bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:07.341821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:07.391253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 385/761] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:07.413794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed286407-a1d7-48e2-9d9f-c436b6284bc7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:07.431055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68991d53-7e7f-4d51-9f0a-78e3b8e8951c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:07.434146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:07.473853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:07.474617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.474902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.475887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.477100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=67316e32d66342d58684c1ee3e5894c1) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:07.477437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:07.480424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:07.489122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 377/762] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:07 2022] GET /identity/v3/role_assignments?user.id=67316e32d66342d58684c1ee3e5894c1&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:07.536659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed286407-a1d7-48e2-9d9f-c436b6284bc7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:07.583472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb2bf3-ff9d-4811-8ff8-a78b3d5185db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:07.584321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2bf3-ff9d-4811-8ff8-a78b3d5185db None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.584608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2bf3-ff9d-4811-8ff8-a78b3d5185db None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.584890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb2bf3-ff9d-4811-8ff8-a78b3d5185db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.597304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb2bf3-ff9d-4811-8ff8-a78b3d5185db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-7efb2bf3-ff9d-4811-8ff8-a78b3d5185db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:07.609431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:07.610624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:07.611724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 386/763] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:07.618835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:07.678673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f866d-e4d0-4a76-a2ba-2b342bd920f3 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.679015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f866d-e4d0-4a76-a2ba-2b342bd920f3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.679233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f866d-e4d0-4a76-a2ba-2b342bd920f3 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.680961 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 387/764] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:07.696007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a002c01-9558-4ba7-b0e2-33bd371b8836 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.696629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a002c01-9558-4ba7-b0e2-33bd371b8836 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.697107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a002c01-9558-4ba7-b0e2-33bd371b8836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.745409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6a002c01-9558-4ba7-b0e2-33bd371b8836 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:07.746952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6a002c01-9558-4ba7-b0e2-33bd371b8836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:07.764394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f21d-4c27-48ad-b80e-08f4ca43831c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:07.797728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:07.809388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:07.811844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:07.811844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:07.811844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:07.811844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:07.811844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:07.812993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:07.839397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 378/765] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:07.853471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:07.861617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 388/766] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:07.895187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a002c01-9558-4ba7-b0e2-33bd371b8836 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:07.909330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:08.011048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5763c9-6fcf-4028-a97b-4582bdc6fda4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.057093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a002c01-9558-4ba7-b0e2-33bd371b8836 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.065830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.066917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.067404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.067941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.092590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.094042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.094735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.096029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.096518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.097295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.098161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.099787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:08.114458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:08.115424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:08.116308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:08.119883 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 389/767] 10.222.0.42 () {56 vars in 1129 bytes} [Thu May 26 21:08:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:08.124654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:08.125574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:08.126439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:08.126439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:08.126439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:08.126439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:08.127911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.129371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 379/768] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:08.136778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.204120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:08.265335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e35bb0-340c-4189-a018-35f1bdd16ccf None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.301720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.302891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.303402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.303957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.304814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:08.305850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:08.306369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.308004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.321379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 380/769] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:08.327567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.373740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9fb8f7-52d2-42b5-abcb-315251039bad None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.457533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.458261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.458483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.458753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.459473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:08.459802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.461277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.473792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 390/770] 10.222.0.42 () {56 vars in 1124 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:08.484739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.501824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:08.505334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e1cc1f-7296-40bc-813a-b93e4696f39c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.574473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.575272 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.575690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.576090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.585618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.587343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.601422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fe87e-bf8a-414c-b26e-72484e33c19d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:08.620554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:08.621628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:08.622725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:08.632655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 381/771] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:08.644799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.656460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.657146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.657430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.657704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.658421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:08.658843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.660278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.666653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:08.675103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 391/772] 10.222.0.42 () {56 vars in 1124 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:08.680707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.813899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e4ffb-cd38-472b-8c03-8e902997a229 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.829790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d46f3d-11d6-47a5-9453-de7c08653153 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:08.870171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.870842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.871176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.871455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.872286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] RBAC: Adding query filter params (interface=public, service_id=c2805a9a473e401bb14e502485064af1, region_id=RegionOne) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:08.872604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.873992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.879632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 392/773] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/endpoints?service_id=c2805a9a473e401bb14e502485064af1&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:08.888864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.893962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:08.894863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:08.895327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:08.897612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:08.899419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:08.908244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:08.909732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:08.943294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 382/774] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:08.949862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:08.977319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:08.993698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:09.065520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c538e67-d768-4dc7-9d4d-3a2581532d89 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:09.083671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66bf70-eef5-4185-8156-268c5fa90b99 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:09.103332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:09.104708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.104988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.105260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.117216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:09.118662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:09.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:09.138555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:09.139482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:09.139482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:09.139482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:09.139482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:09.139482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 383/775] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:09.144525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:09.181091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:09.182306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.182598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.182887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.183586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:09.185024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:09.202644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 393/776] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:08 2022] GET /identity/v3/services => generated 772 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:09.231928 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:09.269277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744f06d-3dfa-4ad9-bc27-6bc9e4f0af3d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:09.313713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:09.314669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.315134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.315571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.316504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:09.316966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:09.318623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:09.343867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 384/777] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:09.352946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:09.490874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc20b7f-b953-48f1-97b6-aa0f13484c0c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:09.526256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:09.534993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d0ec4-e30e-4f65-8b15-16c07dc9fb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:09.536151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d0ec4-e30e-4f65-8b15-16c07dc9fb70 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.541965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d0ec4-e30e-4f65-8b15-16c07dc9fb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.541965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d0ec4-e30e-4f65-8b15-16c07dc9fb70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.545027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d0ec4-e30e-4f65-8b15-16c07dc9fb70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:09.546712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119d0ec4-e30e-4f65-8b15-16c07dc9fb70 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-119d0ec4-e30e-4f65-8b15-16c07dc9fb70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:09.563006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:09.564001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:09.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:09.567374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 394/778] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:09.574324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:09.689870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd435d-a125-4d1c-bd0f-2db50ba9a691 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:09.707998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac596dea-b779-45cd-b81b-ac2f7982e2e6 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.710130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac596dea-b779-45cd-b81b-ac2f7982e2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.710406 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac596dea-b779-45cd-b81b-ac2f7982e2e6 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.711999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 395/779] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:09.717736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84836c74-668f-47c5-bbec-1493248b1588 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.718297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84836c74-668f-47c5-bbec-1493248b1588 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.718865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84836c74-668f-47c5-bbec-1493248b1588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.735198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:09.738631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-84836c74-668f-47c5-bbec-1493248b1588 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:09.740109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-84836c74-668f-47c5-bbec-1493248b1588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:09.753777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:09.754529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.754781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.755060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.755881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:09.756194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:09.757627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:09.779328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 385/780] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:09.785923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:09.824214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 396/781] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:09.847399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84836c74-668f-47c5-bbec-1493248b1588 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:09.950366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fec0c69-4149-4e9a-ab40-9704674ad430 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:09.955924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84836c74-668f-47c5-bbec-1493248b1588 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:09.976375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:09.988336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16873d-c8fd-4e08-bf63-d99673bc2115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:09.994006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16873d-c8fd-4e08-bf63-d99673bc2115 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:09.994006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16873d-c8fd-4e08-bf63-d99673bc2115 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:09.994006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16873d-c8fd-4e08-bf63-d99673bc2115 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/eefd00457d354271867932d336d7f69c/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:09.994006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16873d-c8fd-4e08-bf63-d99673bc2115 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=eefd00457d354271867932d336d7f69c, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:09.994006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16873d-c8fd-4e08-bf63-d99673bc2115 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:10.001046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fc4ab-f078-4f2a-95fe-1ce6be48225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.002019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fc4ab-f078-4f2a-95fe-1ce6be48225e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.002396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fc4ab-f078-4f2a-95fe-1ce6be48225e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.002778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fc4ab-f078-4f2a-95fe-1ce6be48225e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.015038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8fc4ab-f078-4f2a-95fe-1ce6be48225e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-6f8fc4ab-f078-4f2a-95fe-1ce6be48225e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:10.023774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:10.024747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:10.025703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 397/782] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:10.027872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60a8ed-fdf7-49a8-8b92-83bc87417433 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.028415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60a8ed-fdf7-49a8-8b92-83bc87417433 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.028867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60a8ed-fdf7-49a8-8b92-83bc87417433 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.030727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 398/783] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:10.035962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60a8ed-fdf7-49a8-8b92-83bc87417433 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.088184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 386/784] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:08:09 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/eefd00457d354271867932d336d7f69c/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:08:10.089794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85efc2-cb31-4036-a268-b2a14d250e51 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.090145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85efc2-cb31-4036-a268-b2a14d250e51 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.090449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85efc2-cb31-4036-a268-b2a14d250e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.129061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5b85efc2-cb31-4036-a268-b2a14d250e51 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:10.130338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5b85efc2-cb31-4036-a268-b2a14d250e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:10.174724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60a8ed-fdf7-49a8-8b92-83bc87417433 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:10.236262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:10.244891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.245616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.245902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.246181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.246948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:10.247257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:10.248752 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:10.257865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 387/785] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:10.259401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bfcf-922e-4063-86f5-3f79198e7c8c None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.260505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bfcf-922e-4063-86f5-3f79198e7c8c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.260879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bfcf-922e-4063-86f5-3f79198e7c8c None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.262320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 388/786] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:10.269706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52b95c-f6c1-47f2-aefc-5aa8d3667700 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.270010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52b95c-f6c1-47f2-aefc-5aa8d3667700 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.270277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52b95c-f6c1-47f2-aefc-5aa8d3667700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.274333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 399/787] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:10.280767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.295806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-db52b95c-f6c1-47f2-aefc-5aa8d3667700 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:10.295806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-db52b95c-f6c1-47f2-aefc-5aa8d3667700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:10.343718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:10.383863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 389/788] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:10.385627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52b95c-f6c1-47f2-aefc-5aa8d3667700 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.421804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c665154e-b7ae-409a-8dc8-e43f30533cd0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:10.459780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.460589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.460876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.461155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.473274 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:10.475014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:10.503186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:10.504563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:10.505455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:10.505455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:10.505455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:10.505455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:10.505455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 400/789] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:10.508556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.534820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52b95c-f6c1-47f2-aefc-5aa8d3667700 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:10.549299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:10.580032 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.580845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.581160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.581527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] PATH_INFO: `/v3/services/network` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.582203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:10.583686 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:10.597625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:10.598949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 21:08:10.599950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:10.604467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 390/790] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/services/network => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:10.605846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.637710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:10.648921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61c80d-cf96-47ed-b34e-a08efd46ec12 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:10.692625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.697087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.698942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.699292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.719688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:10.727481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97902cdb-dde4-4948-be47-b04b7d6e7228 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:10.728226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:10.729344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:08:10.730170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:10.734811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 401/791] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:10.735731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.770715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.771773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.772210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.772629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.773514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:10.773973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:10.775525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:10.788338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 391/792] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:10.789530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.831467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:10.871053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafebd33-830f-420c-b764-a28b9e77f343 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:10.913085 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc3c6b-ef52-4131-9c29-c54d089f8955 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:10.917007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.917947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.918452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.918945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.919896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] RBAC: Adding query filter params (name=network) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:10.920426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:10.922516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:10.929006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 402/793] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:10.930329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:10.955191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:10.959618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:10.960470 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:10.960753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:10.961032 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:10.961839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:10.962154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:10.964183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:10.978231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 392/794] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:10.979973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.061976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6315c7-35bc-4de1-99a1-f24136f0c5a8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.093389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:11.100874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad1ee7f-c013-41a5-b8db-d018ce45c053 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.108923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.120247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.120247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.120247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.120247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.120639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:11.135587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:11.136580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 21:08:11.137620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:11.141653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 403/795] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:11.143174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.147100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.147790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.148076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.148372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.150103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] RBAC: Adding query filter params (type=network) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:11.150491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.151890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.162153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 393/796] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:11.162153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.243399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:11.272257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e9146-1ed7-4a65-b38c-3633851c57b8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.275366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabc98b-13fe-4031-98a4-60d1a34e2a94 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.313503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.314219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.314500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.314783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.316388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.317462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.317959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.318446 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.319667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:11.320506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.322826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.327517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.329056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.349343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 394/797] 10.222.0.42 () {56 vars in 1135 bytes} [Thu May 26 21:08:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:11.350681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:11.351108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:11.353108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:11.355147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:11.355147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:11.355147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:11.355147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:11.357785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 404/798] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:11.358661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.377562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:11.469805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88750772-31d5-4fda-b4e4-dce083ee9900 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.492464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9ae7a-08b0-41d3-bdb9-2428fa11aef1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.505564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.506266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.506555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.506835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.507652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.509072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.525018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:11.533474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.534360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.534637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.534908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.546293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.547767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:11.565663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:11.566580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:11.567584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:11.567584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:11.567584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:11.567584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:11.567584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 405/799] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:11.569044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.655784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 395/800] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:11.656861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.691923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb96b21-6a52-47d8-be3e-5188e347a4e3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.753880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.756174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.756516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.756799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.758730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:11.759041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.761151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.794112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 406/801] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:11.802438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.842532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:11.872330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463201f6-e7bb-410f-83fb-88f35d203fda None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:11.880214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:11.910238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9e611-7870-4ff5-adf5-f072aca1dd2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:11.911066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9e611-7870-4ff5-adf5-f072aca1dd2f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.911351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9e611-7870-4ff5-adf5-f072aca1dd2f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.911964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9e611-7870-4ff5-adf5-f072aca1dd2f None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.912966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9e611-7870-4ff5-adf5-f072aca1dd2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:11.913276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9e611-7870-4ff5-adf5-f072aca1dd2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:11.916986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9e611-7870-4ff5-adf5-f072aca1dd2f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:11.929623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 396/802] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:11.931096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdd259-1bbc-4d18-9d4f-a713a5361898 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:11.931439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdd259-1bbc-4d18-9d4f-a713a5361898 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:11.932260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bdd259-1bbc-4d18-9d4f-a713a5361898 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:11.933652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 397/803] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:11.939505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdd259-1bbc-4d18-9d4f-a713a5361898 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:11.959684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665b49-dad7-46b7-b010-2f8afbfbc464 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.001380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e70d8-12c2-4b31-a830-eaa7e5e531db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.002235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e70d8-12c2-4b31-a830-eaa7e5e531db None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.002525 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e70d8-12c2-4b31-a830-eaa7e5e531db None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.002807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e70d8-12c2-4b31-a830-eaa7e5e531db None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.010984 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e70d8-12c2-4b31-a830-eaa7e5e531db None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.012492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e70d8-12c2-4b31-a830-eaa7e5e531db None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f75e70d8-12c2-4b31-a830-eaa7e5e531db None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:12.034139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:12.035158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:12.036184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:12.039462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 407/804] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:08:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:12.041838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe9a1-582c-4c7b-9c40-511c843b4e34 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.041838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe9a1-582c-4c7b-9c40-511c843b4e34 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.041838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe9a1-582c-4c7b-9c40-511c843b4e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.059314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-01fbe9a1-582c-4c7b-9c40-511c843b4e34 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:12.060530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-01fbe9a1-582c-4c7b-9c40-511c843b4e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:12.080472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bdd259-1bbc-4d18-9d4f-a713a5361898 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.120158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.120907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.121190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.121479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.123724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:12.129971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.131427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.131577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 408/805] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:12.132442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe9a1-582c-4c7b-9c40-511c843b4e34 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:12.149798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:12.151387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:12.152294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:12.154638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 398/806] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:12.155639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.165531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:12.243572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe9a1-582c-4c7b-9c40-511c843b4e34 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.268547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e03b0-5449-41b2-858b-48c54e8964af None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.281198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.281870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.282168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.282424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.283265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:12.283577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.285083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.302924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 409/807] 10.222.0.42 () {56 vars in 1149 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:12.304919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.305931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.306982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.307511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.308027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.308689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.310063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:12.323131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:12.324229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:12.330630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 399/808] 10.222.0.42 () {56 vars in 1129 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:12.331490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.389451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:12.412142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:12.428402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ba15b-fb9c-4255-ba13-28da76fab0c7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.433029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde3e64-0c13-4d60-9ea6-42ac07237cc7 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.467228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.467918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.468209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.468480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.468992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.469258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:12.469564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.469786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.470072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.470347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.470999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.482053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.483600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.489628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 410/809] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:12.489628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:12.502363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:12.503266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:12.504187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:12.504187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:12.504187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:12.504187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:12.504187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 400/810] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:12.504477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.599491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9097a75e-4fd6-47a2-8d0f-fa38c85a4c51 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.610996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de643d-844a-4990-9acb-c2764043d6fb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.622258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:12.635249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.635951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.636283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.636567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.637275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:12.637586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.638994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.643542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 411/811] 10.222.0.42 () {56 vars in 1124 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:12.644365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.646080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60317799-732b-4814-b9b6-47a5e4ef00a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.646969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60317799-732b-4814-b9b6-47a5e4ef00a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.647277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60317799-732b-4814-b9b6-47a5e4ef00a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.647553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60317799-732b-4814-b9b6-47a5e4ef00a3 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/ededf6d88dee4745978c8566940b2180/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.649553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60317799-732b-4814-b9b6-47a5e4ef00a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=ededf6d88dee4745978c8566940b2180, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.651538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60317799-732b-4814-b9b6-47a5e4ef00a3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.657438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:12.743339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d08e6-9708-4300-94d7-e2356e1f4599 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.761644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 401/812] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:08:12 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/ededf6d88dee4745978c8566940b2180/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:08:12.763116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c13a0-4a27-4e60-bcb8-2b424eb67003 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.763484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c13a0-4a27-4e60-bcb8-2b424eb67003 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.763873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c13a0-4a27-4e60-bcb8-2b424eb67003 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.767804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 402/813] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:12.767804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c13a0-4a27-4e60-bcb8-2b424eb67003 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.781515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb5030-e9ad-49e5-aecd-b317629bb8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.782250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb5030-e9ad-49e5-aecd-b317629bb8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.782616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb5030-e9ad-49e5-aecd-b317629bb8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.782808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb5030-e9ad-49e5-aecd-b317629bb8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.783889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb5030-e9ad-49e5-aecd-b317629bb8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:12.784387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb5030-e9ad-49e5-aecd-b317629bb8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.786515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb5030-e9ad-49e5-aecd-b317629bb8e5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.804616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 412/814] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:12.804616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db314-7318-4641-8626-007eaf1a56b1 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.804616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db314-7318-4641-8626-007eaf1a56b1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.804616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04db314-7318-4641-8626-007eaf1a56b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.822691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a04db314-7318-4641-8626-007eaf1a56b1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:12.823987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a04db314-7318-4641-8626-007eaf1a56b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:12.896274 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:12.898175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58c13a0-4a27-4e60-bcb8-2b424eb67003 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:12.915772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 413/815] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:12.915772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db314-7318-4641-8626-007eaf1a56b1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:12.962239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:12.963644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:12.964058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:12.964339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:12.965061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:12.965434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:12.967321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:12.972980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 403/816] 10.222.0.42 () {56 vars in 1124 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:12.973852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.012425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:13.018516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04db314-7318-4641-8626-007eaf1a56b1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.054031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.054952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.055240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.055516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.063033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:13.064531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:13.079206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:13.080239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:13.081126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:13.085237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 414/817] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:13.085880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.088494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056b1a0-8a0c-4216-8af3-9689b22c24c5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.127410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.129038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.129511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.129963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.142102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:13.147868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:13.150007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:13.151443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:13.152330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 404/818] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:13.153036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.200176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8dace0-a24a-49d9-a7ec-eb90f8354334 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.236343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.237419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.237709 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.237988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.238751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:13.240796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:13.257056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:13.264350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8fc9b-9218-4c40-82d4-6dba39d7138d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.298425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.299082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.299362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.299636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.300454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:13.300762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:13.302221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:13.313809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 405/819] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:13.314611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.351772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 415/820] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:13.351772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.466667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbf6bd-f517-4303-bd00-fbbdb86440ed None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.488039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f149f-5562-4a50-a793-d5285f5749c9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.505666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:13.507824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.508787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.509390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.510002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.511016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:13.511553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:13.513214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:13.523667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 406/821] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:13.528908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.540810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.542651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.542934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.543372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/67316e32d66342d58684c1ee3e5894c1/roles/7c5539dafdbe475babeae5e546b4038c` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.552867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:13.565240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=67316e32d66342d58684c1ee3e5894c1, role_id=7c5539dafdbe475babeae5e546b4038c)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:13.567262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:13.646767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 416/822] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:08:13 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/67316e32d66342d58684c1ee3e5894c1/roles/7c5539dafdbe475babeae5e546b4038c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:08:13.653329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899efa5-af87-4ec9-9d3f-42b5afd422ec None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.691594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.692488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.692775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.693052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.709086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:13.720237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:13.733475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:13.734634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:13.735768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:13.735768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:13.735768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:13.735768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:13.735768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 407/823] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:13.741925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.860129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135796b-3220-420f-8327-a3fea2546b14 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:13.898235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:13.898991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:13.899282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:13.899562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:13.900379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:13.900693 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:13.902137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:13.915471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 417/824] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:13.920817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:13.993122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:14.028254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7bed1-22cf-4ead-97e8-548c80a5d68d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:14.071052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:14.071874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:14.072682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:14.072959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:14.080365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:14.081813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:14.097418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:14.098411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:14.099364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 408/825] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:08:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:14.106115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:14.131718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:14.219077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d49915-11bb-4152-92df-4c26afab6f37 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:14.254583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:14.255602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:14.256111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:14.256732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:14.257792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:14.258330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:14.260526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:14.280713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 418/826] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:08:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:14.286703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:14.393568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec33e6-32a2-42d7-88d5-b3016a0fafe8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:14.422091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:14.430276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:14.434954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:14.434954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:14.434954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:14.504299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:14.505801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:14.613218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 409/827] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:14 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:14.618945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:14.705040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:14.729505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05da6beb-3f58-47ad-91a4-0b412b735892 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:14.765322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:14.766034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:14.766316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:14.766590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:14.778618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:14.780120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:14.798549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:14.799441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:14.800379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:14.800379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:14.800379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:14.800379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:14.800379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 419/828] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:14.805541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:14.835566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:14.928889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724157f7-2e51-4c45-94b5-57d1f3bca2b6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:14.965106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf0490-eff2-4f43-a06a-305cb9685b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:14.965793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0490-eff2-4f43-a06a-305cb9685b00 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:14.966079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0490-eff2-4f43-a06a-305cb9685b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:14.966357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf0490-eff2-4f43-a06a-305cb9685b00 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:14.967081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0490-eff2-4f43-a06a-305cb9685b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:14.967390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0490-eff2-4f43-a06a-305cb9685b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:14.968780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bf0490-eff2-4f43-a06a-305cb9685b00 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:14.981889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 410/829] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:14.987033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.060546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f9387-94dd-4b29-81f6-bcc7dc9df8c7 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.060851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f9387-94dd-4b29-81f6-bcc7dc9df8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.061006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f9387-94dd-4b29-81f6-bcc7dc9df8c7 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.062679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 411/830] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:15.067696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431a4fd-796f-48ec-86ea-667840c637cc None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.068011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431a4fd-796f-48ec-86ea-667840c637cc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.068371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a431a4fd-796f-48ec-86ea-667840c637cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.089126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-a431a4fd-796f-48ec-86ea-667840c637cc None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:15.090811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a431a4fd-796f-48ec-86ea-667840c637cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:15.095905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2604a61-aab4-44bb-babf-95d9e2038564 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:15.125660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:15.132874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:15.133940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.137211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.137211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.142723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:15.144424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:15.160131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:15.161307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:15.162324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:15.164397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 420/831] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:15.170197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.175808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 412/832] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:15.200770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431a4fd-796f-48ec-86ea-667840c637cc None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.223515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:15.306057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d428b8-721a-4f8e-8f39-e3d4df7a2066 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:15.315261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a431a4fd-796f-48ec-86ea-667840c637cc None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:15.353253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:15.354604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.354958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.355306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.356206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:15.356586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:15.356944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:15.357936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.358228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:15.358597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.359064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.371771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:15.378475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 421/833] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:15.379784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:15.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:15.385240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:15.386088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:15.388819 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 413/834] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:15.394471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.437106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:15.486216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:15.489802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e17761-cf1d-4ad8-9f49-d879ffff77ad None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:15.512195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314d59d-c56b-421e-9770-2ce4721897f5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:15.527022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:15.527687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.528001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.528281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.529172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=eefd00457d354271867932d336d7f69c) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:15.529522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:15.531032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:15.543842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 422/835] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/role_assignments?user.id=eefd00457d354271867932d336d7f69c&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:15.559273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac243a28-081d-456c-828c-cf5682264f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:15.560060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac243a28-081d-456c-828c-cf5682264f61 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.560342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac243a28-081d-456c-828c-cf5682264f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.560618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac243a28-081d-456c-828c-cf5682264f61 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.561390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac243a28-081d-456c-828c-cf5682264f61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:15.561704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac243a28-081d-456c-828c-cf5682264f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:15.563179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac243a28-081d-456c-828c-cf5682264f61 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:15.574837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 414/836] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:15.580806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.699494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2291781-54a7-4ee7-ab49-e193cfd37930 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:15.719913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:15.739306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:15.741233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.741233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.741233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.752164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:15.753636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:15.774150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:15.775107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:15.776608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:15.776608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:15.776608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:15.776608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:15.776608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 423/837] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:15.776898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53204f3d-4033-4bcd-b3bd-5590bcfbd295 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.776898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53204f3d-4033-4bcd-b3bd-5590bcfbd295 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.777028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53204f3d-4033-4bcd-b3bd-5590bcfbd295 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.782077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 415/838] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:15.782265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.784386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392d36c-7ce5-4f91-9083-e6ba6e733958 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.784673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392d36c-7ce5-4f91-9083-e6ba6e733958 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.784952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3392d36c-7ce5-4f91-9083-e6ba6e733958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.806054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3392d36c-7ce5-4f91-9083-e6ba6e733958 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:15.807844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3392d36c-7ce5-4f91-9083-e6ba6e733958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:15.887938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 416/839] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:15 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:15.907416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392d36c-7ce5-4f91-9083-e6ba6e733958 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:15.911400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788964f-f5b6-4519-9213-0d442b2391b7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:15.954553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4a5074-e2ea-4139-b2d2-503ae243e041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:15.955218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a5074-e2ea-4139-b2d2-503ae243e041 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.955498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a5074-e2ea-4139-b2d2-503ae243e041 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.955791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4a5074-e2ea-4139-b2d2-503ae243e041 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.956524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a5074-e2ea-4139-b2d2-503ae243e041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:15.956883 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a5074-e2ea-4139-b2d2-503ae243e041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:15.958328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4a5074-e2ea-4139-b2d2-503ae243e041 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:15.970333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 424/840] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:15.971694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfc441-a574-4908-8d07-baa38d6b3c22 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:15.972021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfc441-a574-4908-8d07-baa38d6b3c22 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:15.972304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfc441-a574-4908-8d07-baa38d6b3c22 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:15.973652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 425/841] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:15.976048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfc441-a574-4908-8d07-baa38d6b3c22 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.000038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.006092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.032236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3392d36c-7ce5-4f91-9083-e6ba6e733958 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.067971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a24b2b-04c9-49f5-86c4-5d34f2530390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.068789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a24b2b-04c9-49f5-86c4-5d34f2530390 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.069077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a24b2b-04c9-49f5-86c4-5d34f2530390 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.069359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a24b2b-04c9-49f5-86c4-5d34f2530390 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.070269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a24b2b-04c9-49f5-86c4-5d34f2530390 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.071657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a24b2b-04c9-49f5-86c4-5d34f2530390 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.084197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfc441-a574-4908-8d07-baa38d6b3c22 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-e3a24b2b-04c9-49f5-86c4-5d34f2530390 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:16.084750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:16.085705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:16.086662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 417/842] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:16.087462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837392-b749-4849-89e4-59412dea0010 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.087462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837392-b749-4849-89e4-59412dea0010 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.088013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a837392-b749-4849-89e4-59412dea0010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.106207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-9a837392-b749-4849-89e4-59412dea0010 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:16.107405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9a837392-b749-4849-89e4-59412dea0010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:16.127777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.129127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.129596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.130101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.141505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.142908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:16.162024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:16.163039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:16.164034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 426/843] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:16.165312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.188170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 418/844] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:16 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:16.189609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a837392-b749-4849-89e4-59412dea0010 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.256365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.264294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.272264 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9970f92-c313-46fe-9b70-9c34fa5264d9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.304161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a837392-b749-4849-89e4-59412dea0010 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.312871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.313909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.314210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.314488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.315513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:16.315988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.317651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.331139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 427/845] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:16.331139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.341551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.342229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.342508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.342780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.344005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:16.344624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.346382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.364244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 419/846] 10.222.0.42 () {56 vars in 1135 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:16.369594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.441967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1938df44-8c4b-45d5-bff2-aa74d8857d34 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.475207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0607842b-ce99-45bc-ac5b-e989f4a4b78a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.479390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.480222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.480504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.480777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.491088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:16.498639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:16.499012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:16.500173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:16.501052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 428/847] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:16.501778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.515777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.518307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.519110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.519396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.519773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.520506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:16.520817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.522192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.527394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 420/848] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:16.528035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.647363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430e8896-5d1a-4981-a0ff-746087185ed4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.657507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dacc293-3725-46c6-8ca2-2137ddb56a1c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.696031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.697687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.698159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.698632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.699589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:16.700113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.702009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.714808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 421/849] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:16.720094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.722087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.723312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.723769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.724221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.791769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.816311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.817865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.830549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:16.855878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a650b8e-019d-4b43-9aa1-d3ea8a089b70 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:16.902592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:16.904392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:16.904887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:16.905844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:16.921659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:16.923983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:16.931309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 429/850] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:16.937542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:16.947979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:16.948889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:16.949775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:16.949775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:16.949775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:16.949775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:16.952613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 422/851] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:16.957628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.062191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8840b7-7b52-4e0d-9adf-6323b6db1d4a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.074344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.089532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.094417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d7aa7-dc17-4829-9c9d-b737aa36b015 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.098838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba26989-4d28-4798-be71-f24334086277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.099714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba26989-4d28-4798-be71-f24334086277 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.100123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba26989-4d28-4798-be71-f24334086277 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.100464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba26989-4d28-4798-be71-f24334086277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.112010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba26989-4d28-4798-be71-f24334086277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.113581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba26989-4d28-4798-be71-f24334086277 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-bba26989-4d28-4798-be71-f24334086277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:17.134026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:17.134989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:17.135914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:17.135914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:17.135914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:17.135914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:17.139870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 430/852] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:17.141142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba26989-4d28-4798-be71-f24334086277 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.144481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.146260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.146772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.147271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.148230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:17.148755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.150783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.167190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 423/853] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:17.172546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.253370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba26989-4d28-4798-be71-f24334086277 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.292919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.293447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.293728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.294001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.294791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:17.295116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.296583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.308853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1d557-8563-4941-8abf-edb5afa70e76 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.314141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 431/854] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:17.315548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.343841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.358991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.360574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.361028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.364128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.372088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.374126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:17.393908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:17.394927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:17.395913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 424/855] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:17.401441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.432243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.438135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18777c3-0ee2-4400-9aaf-4a4cc4b79d3f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.474770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.475672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.475940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.476223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.484225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.486838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:17.503286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:17.504295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:17.505201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 432/856] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:17.512145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.534189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe968e85-c76c-457c-b8f5-aedf5e7c4c36 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.569841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.570780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.571059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.571330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.572256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:17.572567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.574200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.593837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 425/857] 10.222.0.42 () {56 vars in 1149 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:17.598315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.608291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.631494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73347bc-5c43-405a-a421-ce9be14af882 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.671595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.672420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.672713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.673317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.674051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:17.674374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.675969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.692092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 433/858] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:17.700067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.707765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:17.723572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b79267-62cb-4ff1-a5f7-7725296053de None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.760769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.761489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.761874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.762252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.823969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddfe3e-5e94-455b-8dcd-96ac4df58df3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:17.835421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.837052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.863989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d9a48-84d3-485f-a2a3-490787e2cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:17.864897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9a48-84d3-485f-a2a3-490787e2cd7c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.865332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9a48-84d3-485f-a2a3-490787e2cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.865875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d9a48-84d3-485f-a2a3-490787e2cd7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.867094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9a48-84d3-485f-a2a3-490787e2cd7c None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=ededf6d88dee4745978c8566940b2180) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:17.867559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9a48-84d3-485f-a2a3-490787e2cd7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:17.869852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d9a48-84d3-485f-a2a3-490787e2cd7c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:17.876998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 434/859] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/role_assignments?user.id=ededf6d88dee4745978c8566940b2180&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:17.883455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbc230-0214-4cd7-bccd-63185625ea17 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.883794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbc230-0214-4cd7-bccd-63185625ea17 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.884032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cbc230-0214-4cd7-bccd-63185625ea17 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.885432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 435/860] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:17.897373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5409-d1b3-4520-a532-6bd9b1ddaaa4 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:17.897698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5409-d1b3-4520-a532-6bd9b1ddaaa4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:17.897953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a5409-d1b3-4520-a532-6bd9b1ddaaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:17.910082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.917476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-609a5409-d1b3-4520-a532-6bd9b1ddaaa4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:17.918904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-609a5409-d1b3-4520-a532-6bd9b1ddaaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:17.948872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:17.984978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 426/861] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:17.991191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.011252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 436/862] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:18.027190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5409-d1b3-4520-a532-6bd9b1ddaaa4 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.112782 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034a018-2695-445d-8319-81ac8eb3da49 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.149708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a5409-d1b3-4520-a532-6bd9b1ddaaa4 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.154421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.155224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.155505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.155801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.169296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.172088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:18.190988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:18.192015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:18.193486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:18.193486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:18.193486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:18.193486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:18.193486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 427/863] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:18.198509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.202600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.203697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.204231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.204743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.205621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.207239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.211147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:18.225160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:18.226151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:18.227005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:18.229484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 437/864] 10.222.0.42 () {56 vars in 1132 bytes} [Thu May 26 21:08:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:18.236622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.241780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:18.340916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79a5e57-4228-4a57-ab03-bd06759169ba None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.362051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743e294-0f3d-4971-ad0f-910c776334bd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.378766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.379828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.380323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.380826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.381777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:18.382291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.383891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.396633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 428/865] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:18.398589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.399263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.399545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.406545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.406545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:18.406545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.406855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.406931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.408191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 438/866] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:18.417247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.487476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:18.523196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aacbeb6-2672-4982-a0ca-11b18cd15e41 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.527132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d574af-eb86-4f3e-969c-232cd0fab70b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.528053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:18.562892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.563694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.564083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.564435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.565224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:18.565610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.565992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.566991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.567101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.567598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.568268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.578422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 439/867] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:18.584982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.586485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:18.604592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:18.605540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:18.606441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 429/868] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:18.612124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.748339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9be04-b269-449e-9ca1-2e7fbda03a37 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.779830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:18.787836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15a502-0bed-4e0f-bb6c-698dbe15436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.788510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a502-0bed-4e0f-bb6c-698dbe15436b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.788794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a502-0bed-4e0f-bb6c-698dbe15436b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.794006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a502-0bed-4e0f-bb6c-698dbe15436b None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.794006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15a502-0bed-4e0f-bb6c-698dbe15436b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:18.794006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15a502-0bed-4e0f-bb6c-698dbe15436b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.794006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15a502-0bed-4e0f-bb6c-698dbe15436b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.804767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 440/869] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:18.811249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:18.884294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33d0c-c927-4207-984f-7f1282e3c3c9 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.884839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33d0c-c927-4207-984f-7f1282e3c3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.884839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33d0c-c927-4207-984f-7f1282e3c3c9 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.886546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 441/870] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:18.892479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bbaf7-0f62-4001-a94f-55dfd8385c14 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.892885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bbaf7-0f62-4001-a94f-55dfd8385c14 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.893268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0bbaf7-0f62-4001-a94f-55dfd8385c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.914636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6e0bbaf7-0f62-4001-a94f-55dfd8385c14 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:18.916885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6e0bbaf7-0f62-4001-a94f-55dfd8385c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:18.923461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930ea5b-0596-4728-8d69-d9c2913040ed None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:18.963796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:18.964497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:18.964780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:18.965049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:18.965879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=67316e32d66342d58684c1ee3e5894c1) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:18.966188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:18.967614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:18.974651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 430/871] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:18 2022] GET /identity/v3/role_assignments?user.id=67316e32d66342d58684c1ee3e5894c1&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:19.009627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 442/872] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:19.033070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:19.058227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:19.159762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc98c7f-7e63-4126-a52c-1c7f871b5816 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:19.198661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fefd8f-cd65-4bc9-a833-991d06c32f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:19.199339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fefd8f-cd65-4bc9-a833-991d06c32f48 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:19.199619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fefd8f-cd65-4bc9-a833-991d06c32f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:19.199972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fefd8f-cd65-4bc9-a833-991d06c32f48 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:19.200718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fefd8f-cd65-4bc9-a833-991d06c32f48 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:19.202088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fefd8f-cd65-4bc9-a833-991d06c32f48 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:19.288582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 431/873] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:19 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:20.168399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da89e09-74b6-4d4b-b1b0-fb3806e38916 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:20.168896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da89e09-74b6-4d4b-b1b0-fb3806e38916 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:20.168896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da89e09-74b6-4d4b-b1b0-fb3806e38916 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:20.170462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 443/874] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:20.175842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc8ac-31ab-48e4-ba3f-dae95f6616eb None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:20.176237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc8ac-31ab-48e4-ba3f-dae95f6616eb None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:20.176436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cc8ac-31ab-48e4-ba3f-dae95f6616eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:20.193552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3e5cc8ac-31ab-48e4-ba3f-dae95f6616eb None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:20.194754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3e5cc8ac-31ab-48e4-ba3f-dae95f6616eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:20.279303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:20.285208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 432/875] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:20.307546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da89e09-74b6-4d4b-b1b0-fb3806e38916 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:20.423078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da89e09-74b6-4d4b-b1b0-fb3806e38916 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:20.444042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:20.462392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:20.463239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:20.463526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:20.463834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:20.464500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:20.465884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:20.479371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:20.481795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:20.483736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 444/876] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:20.488890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cc8ac-31ab-48e4-ba3f-dae95f6616eb None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:20.594992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cc8ac-31ab-48e4-ba3f-dae95f6616eb None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:20.630070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496cc783-eeec-4287-9d3e-80c8d40f0f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:20.631094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cc783-eeec-4287-9d3e-80c8d40f0f79 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:20.631609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cc783-eeec-4287-9d3e-80c8d40f0f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:20.632123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496cc783-eeec-4287-9d3e-80c8d40f0f79 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:20.633059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cc783-eeec-4287-9d3e-80c8d40f0f79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:20.633625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cc783-eeec-4287-9d3e-80c8d40f0f79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:20.635351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496cc783-eeec-4287-9d3e-80c8d40f0f79 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:20.640924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 433/877] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:20.651012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:20.770723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594155c-a7b0-4d3c-be3b-34f144ba49c4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:20.809377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035c73a-213e-467c-b39c-83b559b269b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:20.810046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c73a-213e-467c-b39c-83b559b269b8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:20.810330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c73a-213e-467c-b39c-83b559b269b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:20.810605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035c73a-213e-467c-b39c-83b559b269b8 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:20.811316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c73a-213e-467c-b39c-83b559b269b8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:20.811624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c73a-213e-467c-b39c-83b559b269b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:20.813046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035c73a-213e-467c-b39c-83b559b269b8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:20.817992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 445/878] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:20.950800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c851365-04a4-40f0-b9a0-81fe81a840d8 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:20.951160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c851365-04a4-40f0-b9a0-81fe81a840d8 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:20.951370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c851365-04a4-40f0-b9a0-81fe81a840d8 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:20.953270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 434/879] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:20.959270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d1bc-177d-4c2f-ab74-e863893e84fd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:20.959561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d1bc-177d-4c2f-ab74-e863893e84fd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:20.959800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0d1bc-177d-4c2f-ab74-e863893e84fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:20.978781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7de0d1bc-177d-4c2f-ab74-e863893e84fd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:20.980058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7de0d1bc-177d-4c2f-ab74-e863893e84fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:21.028789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:21.055916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 446/880] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:20 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:21.080145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c851365-04a4-40f0-b9a0-81fe81a840d8 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:21.148122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:21.188341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c851365-04a4-40f0-b9a0-81fe81a840d8 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:21.229920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:21.230939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.231430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.231971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.232909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:21.234493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:21.398728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8772-1014-41e4-a2c4-718aec1afdab None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.399803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8772-1014-41e4-a2c4-718aec1afdab None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.400380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e8772-1014-41e4-a2c4-718aec1afdab None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.402461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 447/881] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:21.402946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 435/882] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:21 2022] POST /identity/v3/services => generated 269 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:21.408939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9104-4d86-4866-b867-d4bfc5d24e24 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.409547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9104-4d86-4866-b867-d4bfc5d24e24 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.420647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c9104-4d86-4866-b867-d4bfc5d24e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.429429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1f4c9104-4d86-4866-b867-d4bfc5d24e24 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:21.431039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1f4c9104-4d86-4866-b867-d4bfc5d24e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:21.519078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 448/883] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:21.543430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:21.554465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbce22-7d99-4fb2-b9c9-7fe30a7fe45e None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.554784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbce22-7d99-4fb2-b9c9-7fe30a7fe45e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.555023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cbce22-7d99-4fb2-b9c9-7fe30a7fe45e None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.558545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 449/884] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:21.563002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc7bb2-b549-465c-b282-4f2a9d1d5fe4 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.563285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc7bb2-b549-465c-b282-4f2a9d1d5fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.563784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc7bb2-b549-465c-b282-4f2a9d1d5fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.582961 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3fbc7bb2-b549-465c-b282-4f2a9d1d5fe4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:21.584517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3fbc7bb2-b549-465c-b282-4f2a9d1d5fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:21.665991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50d9bb-c866-43b8-8bf3-1248eeb1594c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:21.675763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 450/885] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:21 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:21.692578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc7bb2-b549-465c-b282-4f2a9d1d5fe4 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:21.697569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:21.714389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:21.715218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.715508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.715818 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.726812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:21.733633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:21.735185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:21.736151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:21.737096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 436/886] 10.222.0.42 () {56 vars in 1120 bytes} [Thu May 26 21:08:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:21.744960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:21.809298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc7bb2-b549-465c-b282-4f2a9d1d5fe4 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:21.846319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:21.847496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.848008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.848483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.849338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:21.850976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:21.860957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbd3e21-0786-43ae-9bf6-44c556dff391 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:21.864195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:21.865150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:21.866095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:21.874133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 451/887] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:21.875373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:21.897996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:21.898676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:21.898957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:21.899230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:21.901151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:21.901151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:21.902420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:21.916591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 437/888] 10.222.0.42 () {56 vars in 1115 bytes} [Thu May 26 21:08:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:21.917901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:21.988182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:21.996073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:22.004054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d18f6-7bb7-41b7-9c4b-24672bc400ab None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.038358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702675f-48b0-465f-82e1-d15dde90df32 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.049281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.050047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.050338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.050724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.051457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:22.052039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.053723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.059524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 452/889] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:22.065112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:22.076026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.076831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.077114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.077390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.089588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.091454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:22.110418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:22.111731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:22.112882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:22.112882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:22.112882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:22.112882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:22.119369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 438/890] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:22.120299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:22.185622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63aa11-4e42-4502-9613-ef28a87ab2e5 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.223065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.223725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:22.225667 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.226180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.226660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.227240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b14d89e-7062-4b32-a70b-937d953adac3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.227687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:22.228146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.229626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.239618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 453/891] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:22 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:22.240653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:22.267336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.268345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.268811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.269299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.270292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:22.271008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:22.271914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.273533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.286427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 439/892] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:22.292177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:22.357132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44dc2e-145c-47f9-b260-d6fa712de902 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.393711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.394488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.394823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.395200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.396062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] RBAC: Adding query filter params (interface=public, service_id=fcf4e3bec17f43fba22eb455798d2fe0, region_id=RegionOne) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:22.396445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.398108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.403859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffcd9db-753f-42ea-9bbd-e050a8f39e0a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.407119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 454/893] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:08:22 2022] GET /identity/v3/endpoints?service_id=fcf4e3bec17f43fba22eb455798d2fe0&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:22.409541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:22.442934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.444344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.444809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.445517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.454559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.456410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:22.472991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:22.473985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:22.474954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:22.477417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 440/894] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:08:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:22.483851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:22.506188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:22.513519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:22.526538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf6596-4628-485c-8cdc-06726b0cee12 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.563173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.563497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.563908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.564286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.565168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.566671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.580169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 455/895] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:22 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:08:22.600980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46664bb-98d4-4e4c-938e-fed8681cf2a2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.645655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.645655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.645655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.645655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.646113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:22.646113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.646113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:22.668242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 441/896] 10.222.0.42 () {56 vars in 1149 bytes} [Thu May 26 21:08:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:22.676562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:22.789586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c0208a-d9ef-4d36-9892-0dcca5c5be6a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:22.826288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee547d-9178-4a27-abe5-ca30d145a4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:22.827302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee547d-9178-4a27-abe5-ca30d145a4fc None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:22.827819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee547d-9178-4a27-abe5-ca30d145a4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:22.828365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee547d-9178-4a27-abe5-ca30d145a4fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:22.851569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:22.898518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee547d-9178-4a27-abe5-ca30d145a4fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:22.900295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee547d-9178-4a27-abe5-ca30d145a4fc None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:23.008742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 456/897] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:22 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:23.015295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:23.097182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:23.126608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bb846-9fae-4a1c-b7dc-caf84c4a9758 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:23.152540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4f6b-df34-4bd9-83a2-d60565f3c9b4 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.152822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4f6b-df34-4bd9-83a2-d60565f3c9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.153031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c4f6b-df34-4bd9-83a2-d60565f3c9b4 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.154644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 457/898] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:23.159782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575e39-59b8-4ecd-bcd6-b4e6229b7c0d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.160087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575e39-59b8-4ecd-bcd6-b4e6229b7c0d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.160355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44575e39-59b8-4ecd-bcd6-b4e6229b7c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.164458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:23.165264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.166250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.166479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.177719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:23.178033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-44575e39-59b8-4ecd-bcd6-b4e6229b7c0d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:23.179163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:23.179286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-44575e39-59b8-4ecd-bcd6-b4e6229b7c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:23.199152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:23.200148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:23.201069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:23.201069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:23.201069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:23.201069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:23.201069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 442/899] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:23.207367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:23.241306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:23.264934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 458/900] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:23.287556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44575e39-59b8-4ecd-bcd6-b4e6229b7c0d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:23.329175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a946a-da42-4a45-8e3d-9bcb81c0c745 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:23.366362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:23.377578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:23.378499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.378781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.379531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.380555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:23.380872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:23.382505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:23.398909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44575e39-59b8-4ecd-bcd6-b4e6229b7c0d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:23.399221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 443/901] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:23.407817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:23.440299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:23.441132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.441486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.441841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.442656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:23.444155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:23.514655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643b87d-bf8d-4909-87c6-3ae977d3726e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:23.529514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 459/902] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:23 2022] POST /identity/v3/services => generated 241 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:08:23.579633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3ca86-f14b-432c-a167-98f54925cc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:23.580497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3ca86-f14b-432c-a167-98f54925cc8b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.580781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3ca86-f14b-432c-a167-98f54925cc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.581055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3ca86-f14b-432c-a167-98f54925cc8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.589736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3ca86-f14b-432c-a167-98f54925cc8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:23.591415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3ca86-f14b-432c-a167-98f54925cc8b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-c9a3ca86-f14b-432c-a167-98f54925cc8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:23.608462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:23.609479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:23.610390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 444/903] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:23.615975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:23.668008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:23.735342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d24bb-29c1-4cdf-8a16-5dbf32fb52e1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:23.776419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:23.777146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.777437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.777712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.778509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:23.778839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:23.780299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:23.795027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 460/904] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:23.800674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7dcee-bd0c-4124-88ca-97976e388f44 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.800936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:23.801058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7dcee-bd0c-4124-88ca-97976e388f44 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.801234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7dcee-bd0c-4124-88ca-97976e388f44 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.802576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 445/905] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:23.808187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5b6d-7f5c-42bf-94d6-c90beb609149 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.808476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5b6d-7f5c-42bf-94d6-c90beb609149 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.808758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a5b6d-7f5c-42bf-94d6-c90beb609149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.826557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3f9a5b6d-7f5c-42bf-94d6-c90beb609149 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:23.827789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3f9a5b6d-7f5c-42bf-94d6-c90beb609149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:23.873567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:23.904060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 446/906] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:23.920511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052504d-271f-4f06-a20d-007da0097c58 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:23.922418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:23.926700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5b6d-7f5c-42bf-94d6-c90beb609149 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:23.966230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:23.967332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:23.967856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:23.968333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:23.969399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] RBAC: Adding query filter params (role.id=13a9265b6efa4d9da45dade8430e7d87, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=67316e32d66342d58684c1ee3e5894c1) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:23.969918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:23.971623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:23.983898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 461/907] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:23 2022] GET /identity/v3/role_assignments?user.id=67316e32d66342d58684c1ee3e5894c1&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=13a9265b6efa4d9da45dade8430e7d87 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:24.046553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a5b6d-7f5c-42bf-94d6-c90beb609149 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:24.086325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:24.087164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:24.087477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:24.087787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:24.088473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:24.089852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:24.103035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:24.104592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:24.105426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 447/908] 10.222.0.42 () {56 vars in 1132 bytes} [Thu May 26 21:08:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:24.113361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:24.224826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3da70-760f-43f6-9e0e-1fbbf86f0de4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:24.260258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:24.260978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:24.261267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:24.261545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:24.262261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:24.262569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:24.263973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:24.269227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 462/909] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:24.276860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:24.325781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:24.388099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf3b5f8-d91d-44ba-be10-f11ff782b6a2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:24.423651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:24.424349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:24.424634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:24.424910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:24.425619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:24.425931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:24.427299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:24.433453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 448/910] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:24.440854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:24.475422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:24.554617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e842f1-a6f1-47c2-b7fa-5556d88e92a4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:24.590751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1707a7-5db5-49e9-8ea4-3efbf5c0f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:24.591416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1707a7-5db5-49e9-8ea4-3efbf5c0f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:24.591695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1707a7-5db5-49e9-8ea4-3efbf5c0f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:24.591999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1707a7-5db5-49e9-8ea4-3efbf5c0f6c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:24.592795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1707a7-5db5-49e9-8ea4-3efbf5c0f6c5 None admin] RBAC: Adding query filter params (interface=public, service_id=68a7d950e34649719d4c76085ecc65bb, region_id=RegionOne) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:24.593107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1707a7-5db5-49e9-8ea4-3efbf5c0f6c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:24.594496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1707a7-5db5-49e9-8ea4-3efbf5c0f6c5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:24.600359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 463/911] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:08:24 2022] GET /identity/v3/endpoints?service_id=68a7d950e34649719d4c76085ecc65bb&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:24.605830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:24.714741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435fb4c-5a0f-4d5c-ab84-6dfe0b6b1a22 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:24.716376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:24.749621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476d40fd-65a0-42fe-bf2e-41d30834a8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:24.750211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d40fd-65a0-42fe-bf2e-41d30834a8ff None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:24.750498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d40fd-65a0-42fe-bf2e-41d30834a8ff None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:24.750773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476d40fd-65a0-42fe-bf2e-41d30834a8ff None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:24.751456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d40fd-65a0-42fe-bf2e-41d30834a8ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:24.752849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476d40fd-65a0-42fe-bf2e-41d30834a8ff None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:24.760024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 449/912] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:24 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:08:25.006566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d147-f205-412e-ba83-3b4542a62ec0 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.007170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d147-f205-412e-ba83-3b4542a62ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.007666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769d147-f205-412e-ba83-3b4542a62ec0 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.009559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 464/913] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:25.015197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf074-4cfc-4af0-84ee-d3e655d1bffa None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.015495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf074-4cfc-4af0-84ee-d3e655d1bffa None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.015808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cf074-4cfc-4af0-84ee-d3e655d1bffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.033519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-9c4cf074-4cfc-4af0-84ee-d3e655d1bffa None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:25.034715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9c4cf074-4cfc-4af0-84ee-d3e655d1bffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:25.106348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 450/914] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:25.128619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769d147-f205-412e-ba83-3b4542a62ec0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:25.231873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:25.236859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769d147-f205-412e-ba83-3b4542a62ec0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:25.272915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:25.273634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.273916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.274189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.274844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:25.276423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:25.288763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:25.289775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 21:08:25.291153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:25.295467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 465/915] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:25.296844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cf074-4cfc-4af0-84ee-d3e655d1bffa None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:25.398077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cf074-4cfc-4af0-84ee-d3e655d1bffa None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:25.399671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:25.434441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:25.435121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.435405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.435682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.436435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:25.436744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:25.438111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:25.443783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 451/916] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:25.449430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:25.558030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a6b7c-1b6e-4329-8966-122b72661fc7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:25.594368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf32194-560c-4f9b-ae29-e55ec1ec1be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:25.595041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32194-560c-4f9b-ae29-e55ec1ec1be2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.595325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32194-560c-4f9b-ae29-e55ec1ec1be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.595599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf32194-560c-4f9b-ae29-e55ec1ec1be2 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.596347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32194-560c-4f9b-ae29-e55ec1ec1be2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:25.596653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32194-560c-4f9b-ae29-e55ec1ec1be2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:25.598111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf32194-560c-4f9b-ae29-e55ec1ec1be2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:25.607853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 466/917] 10.222.0.42 () {56 vars in 1114 bytes} [Thu May 26 21:08:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:25.609538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:25.713600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7f8ad-332c-45cf-909a-e5d56af916a4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:25.749614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9e871-cccd-4cfa-95b8-4076485b665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:25.750307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e871-cccd-4cfa-95b8-4076485b665f None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.750595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e871-cccd-4cfa-95b8-4076485b665f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.750874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e871-cccd-4cfa-95b8-4076485b665f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.751657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9e871-cccd-4cfa-95b8-4076485b665f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:25.753112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9e871-cccd-4cfa-95b8-4076485b665f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:25.848524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 452/918] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:25.949742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e904d8e-e845-4044-9a13-acdef60c944f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.950217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e904d8e-e845-4044-9a13-acdef60c944f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.950606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e904d8e-e845-4044-9a13-acdef60c944f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.952649 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 467/919] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:25.958656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadaf5f-b2a3-47eb-86ee-c5840c629841 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:25.959105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadaf5f-b2a3-47eb-86ee-c5840c629841 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:25.959497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadaf5f-b2a3-47eb-86ee-c5840c629841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:25.979411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ecadaf5f-b2a3-47eb-86ee-c5840c629841 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:25.980674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ecadaf5f-b2a3-47eb-86ee-c5840c629841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:25.994300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:26.064692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 453/920] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:25 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:26.088548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e904d8e-e845-4044-9a13-acdef60c944f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.115053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:26.201609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e904d8e-e845-4044-9a13-acdef60c944f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:26.236330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a517e-64c1-4bac-9ac4-24987ef5dc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:26.237139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a517e-64c1-4bac-9ac4-24987ef5dc7d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.237421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a517e-64c1-4bac-9ac4-24987ef5dc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.237697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a517e-64c1-4bac-9ac4-24987ef5dc7d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.238348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a517e-64c1-4bac-9ac4-24987ef5dc7d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:26.239789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a517e-64c1-4bac-9ac4-24987ef5dc7d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-fb5a517e-64c1-4bac-9ac4-24987ef5dc7d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:26.253009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:26.255354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:26.256460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 468/921] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:26.260533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadaf5f-b2a3-47eb-86ee-c5840c629841 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.331734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad160e-f2fe-4762-b0eb-ac9833c01d3f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.332090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad160e-f2fe-4762-b0eb-ac9833c01d3f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.332331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad160e-f2fe-4762-b0eb-ac9833c01d3f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.334822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 469/922] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:26.340245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc8cdf-2811-47e7-bc15-56e86eeb6f2d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.340532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc8cdf-2811-47e7-bc15-56e86eeb6f2d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.340813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedc8cdf-2811-47e7-bc15-56e86eeb6f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.363311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-eedc8cdf-2811-47e7-bc15-56e86eeb6f2d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:26.364761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-eedc8cdf-2811-47e7-bc15-56e86eeb6f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:26.371114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadaf5f-b2a3-47eb-86ee-c5840c629841 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:26.408251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:26.409061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.409242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.409466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.410182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:26.410494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:26.411909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:26.417393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 454/923] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:26.423252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.433594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:26.443305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 470/924] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:26.466577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc8cdf-2811-47e7-bc15-56e86eeb6f2d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.484360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:26.531318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3efd8-0c50-407b-a795-97cc807ca71b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:26.567168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:26.567907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.568196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.568480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.569197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:26.569509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:26.571165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:26.573558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedc8cdf-2811-47e7-bc15-56e86eeb6f2d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:26.581736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 455/925] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:26.582732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.612886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:26.614119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.614589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.615010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.627770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:26.637537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:26.648308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 471/926] 10.222.0.42 () {56 vars in 1120 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:26.649376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.661795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:26.695206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1dbc8-76e8-4ffc-ba48-60b37b5af10e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:26.741034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:26.742248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.742806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.743094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.744288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] RBAC: Adding query filter params (interface=public, service_id=10430a895e5642d28a55000cb7674950, region_id=RegionOne) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:26.744610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:26.746713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:26.759843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 456/927] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/endpoints?service_id=10430a895e5642d28a55000cb7674950&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:26.762377 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.765146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:26.794340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e6139-63ba-435e-95e5-9dfe885ea703 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:26.837266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:26.838551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.838832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.839103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.839904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:26.840237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:26.841618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:26.852309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 472/928] 10.222.0.42 () {56 vars in 1115 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:26.857685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:26.904199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05ff99-d632-4a8d-9c76-bd53455fdb6d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:26.953023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:26.954227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:26.955013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:26.955502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:26.956422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:26.958488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:26.960132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:26.967851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 457/929] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/services => generated 1497 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:08:26.974639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d7e25-e283-4fc7-bd85-9c5a006421bb None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.017965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02243b9-b2e7-4635-bb0a-557281aa1bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.019177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02243b9-b2e7-4635-bb0a-557281aa1bfc None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.019621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02243b9-b2e7-4635-bb0a-557281aa1bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.020067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02243b9-b2e7-4635-bb0a-557281aa1bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.028942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.034300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02243b9-b2e7-4635-bb0a-557281aa1bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.036256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02243b9-b2e7-4635-bb0a-557281aa1bfc None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-d02243b9-b2e7-4635-bb0a-557281aa1bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:27.057039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:27.058204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:27.059332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:27.059332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:27.059332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:27.059332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:27.061465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 473/930] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:27.067445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.092902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac536b94-359a-4926-a932-57b2973d5cd9 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.093193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac536b94-359a-4926-a932-57b2973d5cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.093481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac536b94-359a-4926-a932-57b2973d5cd9 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.095148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 474/931] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:27.100654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3472-b3f5-4cdd-bd16-77f49015598a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.100947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3472-b3f5-4cdd-bd16-77f49015598a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.101229 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf3472-b3f5-4cdd-bd16-77f49015598a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.121875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-91bf3472-b3f5-4cdd-bd16-77f49015598a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:27.122716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-91bf3472-b3f5-4cdd-bd16-77f49015598a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:27.205024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 475/932] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:27.209087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9606fea-8cc2-4f0c-9006-6ef2a5dd0481 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.227193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf3472-b3f5-4cdd-bd16-77f49015598a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.249970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.251219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.251661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.252133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.253516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:27.254118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.255944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.271054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 458/933] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:27.276871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.341059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.351846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf3472-b3f5-4cdd-bd16-77f49015598a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.378656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.389823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd1d21-d3f1-4f26-8928-39f2557fea24 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.396565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.397660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.398135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.399038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.399991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.402002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:27.419674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:27.420697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 21:08:27.421527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:27.424612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 476/934] 10.222.0.42 () {56 vars in 1132 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:27.430122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.436545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.437378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.437664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.437946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.446550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.448785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:27.466153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:27.467129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:27.468134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 459/935] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:27.474933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.566683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7951e-d43b-48b3-a95c-9614d28fbbf8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.596809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16deca-de46-4011-bc3e-5b04a25bacb2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.606713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.607765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.608353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.608847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.609930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:27.610452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.612124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.617641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.620000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 477/936] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:27.625361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.640662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.640662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.640662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.640662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.640983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:27.640983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.642919 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.668598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 460/937] 10.222.0.42 () {56 vars in 1149 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:27.672708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.685199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.754876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb625a-ae09-4e34-bc06-fd9b6ae51bca None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.789978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4ad5e4-49ea-4fe2-8b43-4d93ac1bb098 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.799218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.799918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.800200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.800475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.801210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:27.801574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.803468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.811795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 478/938] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:27.816878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.834573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.835536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.835921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.836363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.848244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.849795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:27.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:27.869638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:27.870520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:27.870520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:27.870520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:27.870520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:27.870520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 461/939] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:27.876601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:27.908598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.948802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92815110-1a95-46e3-b5ed-bae95c541d2e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.965487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:27.990633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65994ae2-5f1c-4b05-8e45-9f7777c67521 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:27.991644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af98f8-09e7-4a75-a150-4c12560111c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:27.992489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af98f8-09e7-4a75-a150-4c12560111c8 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:27.992880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af98f8-09e7-4a75-a150-4c12560111c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:27.993504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af98f8-09e7-4a75-a150-4c12560111c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:27.994942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af98f8-09e7-4a75-a150-4c12560111c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:27.998027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af98f8-09e7-4a75-a150-4c12560111c8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.044683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:28.045502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.045724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.046023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.046814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:28.047131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:28.048572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.060836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 462/940] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:28.066199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:28.088282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 479/941] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:28.172657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:28.186829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e447a0-4d82-4e9b-9cd8-cfe0e0ce237d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:28.197500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7ccda-34a6-43b5-9829-4449f41eb156 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.197853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7ccda-34a6-43b5-9829-4449f41eb156 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.198063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7ccda-34a6-43b5-9829-4449f41eb156 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.199535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 480/942] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:28.205825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab7af7-058d-4d1d-bb31-72df21275995 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.206159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab7af7-058d-4d1d-bb31-72df21275995 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.206378 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab7af7-058d-4d1d-bb31-72df21275995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.227684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-bdab7af7-058d-4d1d-bb31-72df21275995 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:28.229855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bdab7af7-058d-4d1d-bb31-72df21275995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:28.239721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:28.240668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.240959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.241233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.252319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:28.255170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:28.280192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:28.281187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:28.282076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:28.285308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 463/943] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:28.301335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:28.321600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 481/944] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:28.344871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab7af7-058d-4d1d-bb31-72df21275995 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:28.418620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287b10f-73ac-491f-a5fe-3faa90654e7f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:28.455990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:28.456820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.457113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.457396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.458195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:28.458511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:28.459982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.472142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:28.473696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 464/945] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:28.476678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab7af7-058d-4d1d-bb31-72df21275995 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:28.480309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:28.485092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:28.516715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:28.517386 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.517787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.517999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.518869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:28.520684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.521966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING keystone.server.flask.common [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 21:08:28.584191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.sql.core [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 21:08:28.584191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 21:08:28.584191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [parameters: {'id': 'e3d5a8d6364c4a85b4c6c9cc1b6589ee', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 21:08:28.584191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80756) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3d5a8d6364c4a85b4c6c9cc1b6589ee.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3d5a8d6364c4a85b4c6c9cc1b6589ee. May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result = self._query(query) May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application conn.query(q) May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 21:08:28.594630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result.read() May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 21:08:28.595507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application next(self.gen) May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application yield resource May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application next(self.gen) May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.session.rollback() May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.force_reraise() May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise self.value May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application session.commit() May 26 21:08:28.596838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._prepare_impl() May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.session.flush() May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._flush(objects) May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application compat.raise_( May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application flush_context.execute() May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 21:08:28.597807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rec.execute(self) May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 21:08:28.599181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result = self._query(query) May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application conn.query(q) May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result.read() May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 21:08:28.600046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [parameters: {'id': 'e3d5a8d6364c4a85b4c6c9cc1b6589ee', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:28.600932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:28.602404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3d5a8d6364c4a85b4c6c9cc1b6589ee. May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 482/946] 10.222.0.42 () {60 vars in 1140 bytes} [Thu May 26 21:08:28 2022] POST /identity/v3/projects => generated 262 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:28.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:28.617315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f7a68-039e-447b-8c09-2ff5506c7cf8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:28.670434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:28.671951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.672763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.673224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/67316e32d66342d58684c1ee3e5894c1/roles/13a9265b6efa4d9da45dade8430e7d87` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.676478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=67316e32d66342d58684c1ee3e5894c1, role_id=13a9265b6efa4d9da45dade8430e7d87)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:28.679036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.717615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd43039-4717-4855-8199-3742d1b984f7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:28.753617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5a7019-e5b8-4e39-984a-3685f4431028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:28.754409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7019-e5b8-4e39-984a-3685f4431028 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.754691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7019-e5b8-4e39-984a-3685f4431028 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.754962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5a7019-e5b8-4e39-984a-3685f4431028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.762658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a7019-e5b8-4e39-984a-3685f4431028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:28.764369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5a7019-e5b8-4e39-984a-3685f4431028 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.769805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 465/947] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:08:28 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/67316e32d66342d58684c1ee3e5894c1/roles/13a9265b6efa4d9da45dade8430e7d87 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-ad5a7019-e5b8-4e39-984a-3685f4431028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:28.793089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:28.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:28.795230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:28.799481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 483/948] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:28.800932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:28.876017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:28.921049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118962d8-4a5b-40b7-ad43-d0d3cbfc6752 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:28.963289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e356c-de85-439a-8cfc-425cd35e9ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:28.964186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e356c-de85-439a-8cfc-425cd35e9ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:28.964620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e356c-de85-439a-8cfc-425cd35e9ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:28.964997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e356c-de85-439a-8cfc-425cd35e9ca5 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:28.965933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e356c-de85-439a-8cfc-425cd35e9ca5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:28.966342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e356c-de85-439a-8cfc-425cd35e9ca5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:28.968028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e356c-de85-439a-8cfc-425cd35e9ca5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:28.990637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 466/949] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:28 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:29.369680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5d20-835e-4dcc-868b-9c417e26cac9 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:29.369987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5d20-835e-4dcc-868b-9c417e26cac9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:29.370244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d5d20-835e-4dcc-868b-9c417e26cac9 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:29.374828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 484/950] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:29.380591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a72f2-40ca-440c-bf4c-23dad6f3fe3d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:29.380953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a72f2-40ca-440c-bf4c-23dad6f3fe3d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:29.381303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a72f2-40ca-440c-bf4c-23dad6f3fe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:29.398727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-461a72f2-40ca-440c-bf4c-23dad6f3fe3d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:29.400506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-461a72f2-40ca-440c-bf4c-23dad6f3fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:29.470692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 467/951] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:29.493770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d5d20-835e-4dcc-868b-9c417e26cac9 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:29.511584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:29.605269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d5d20-835e-4dcc-868b-9c417e26cac9 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:29.641143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:29.642453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:29.642926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:29.643400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:29.644286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:29.645968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:29.659147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:29.660337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:29.661325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:29.664102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 485/952] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:29.670538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a72f2-40ca-440c-bf4c-23dad6f3fe3d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:29.727485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:29.772887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a72f2-40ca-440c-bf4c-23dad6f3fe3d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:29.808299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:29.808972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:29.809257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:29.809539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:29.810250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:29.810563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:29.812023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:29.817224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 468/953] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:29.823954 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:29.910014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:29.933904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3823de-aecb-497f-9ad1-80acc941b213 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:29.970271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f987223-bb5c-4ba6-a039-bd6509449160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:29.971040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f987223-bb5c-4ba6-a039-bd6509449160 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:29.971325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f987223-bb5c-4ba6-a039-bd6509449160 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:29.971601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f987223-bb5c-4ba6-a039-bd6509449160 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:29.972346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f987223-bb5c-4ba6-a039-bd6509449160 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:29.972658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f987223-bb5c-4ba6-a039-bd6509449160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:29.974045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f987223-bb5c-4ba6-a039-bd6509449160 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:29.980838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 486/954] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:29.986601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:30.092963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba7af5-55cf-4f76-8cc6-eeb3a2bae8dc None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:30.111313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:30.128448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f031a-31c9-4c7e-a216-7020e616a27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:30.129109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f031a-31c9-4c7e-a216-7020e616a27d None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:30.129385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f031a-31c9-4c7e-a216-7020e616a27d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:30.129661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f031a-31c9-4c7e-a216-7020e616a27d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:30.130412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f031a-31c9-4c7e-a216-7020e616a27d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:30.131783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f031a-31c9-4c7e-a216-7020e616a27d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:30.202240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 469/955] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:30.468776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377862c-2dc9-4048-add0-dc66d87c680c None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:30.469140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377862c-2dc9-4048-add0-dc66d87c680c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:30.469515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a377862c-2dc9-4048-add0-dc66d87c680c None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:30.471254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 487/956] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:30.476478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba36364-b62c-4e9a-ac51-9ed481661fea None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:30.476932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba36364-b62c-4e9a-ac51-9ed481661fea None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:30.477219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba36364-b62c-4e9a-ac51-9ed481661fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:30.495790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6ba36364-b62c-4e9a-ac51-9ed481661fea None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:30.497445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6ba36364-b62c-4e9a-ac51-9ed481661fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:30.575116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 470/957] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:30.596323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377862c-2dc9-4048-add0-dc66d87c680c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:30.720912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a377862c-2dc9-4048-add0-dc66d87c680c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:30.757175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:30.757982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:30.758271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:30.758548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:30.759730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:30.761147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:30.773451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:30.774492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:30.776746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 488/958] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:30.781096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba36364-b62c-4e9a-ac51-9ed481661fea None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:30.891997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:30.894997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba36364-b62c-4e9a-ac51-9ed481661fea None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:30.931897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8c6de-34af-407c-a192-c4b3ab82ba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:30.933019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8c6de-34af-407c-a192-c4b3ab82ba12 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:30.933510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8c6de-34af-407c-a192-c4b3ab82ba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:30.933988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8c6de-34af-407c-a192-c4b3ab82ba12 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:30.934936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8c6de-34af-407c-a192-c4b3ab82ba12 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:30.935449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8c6de-34af-407c-a192-c4b3ab82ba12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:30.937129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8c6de-34af-407c-a192-c4b3ab82ba12 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:30.943407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 471/959] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:30.950548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:30.961369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:31.070216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce201ec-7550-46e8-976f-ccdeb1bcb890 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:31.107423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8326e-003c-4c7e-b67b-bd9b7e5ee370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:31.108216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8326e-003c-4c7e-b67b-bd9b7e5ee370 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.108577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8326e-003c-4c7e-b67b-bd9b7e5ee370 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.108905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8326e-003c-4c7e-b67b-bd9b7e5ee370 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.109745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8326e-003c-4c7e-b67b-bd9b7e5ee370 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:31.110062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8326e-003c-4c7e-b67b-bd9b7e5ee370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:31.111464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8326e-003c-4c7e-b67b-bd9b7e5ee370 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:31.117039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 489/960] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:31.169985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ccb05-9c04-4df1-a3b8-e950c6d8b7b4 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.170667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ccb05-9c04-4df1-a3b8-e950c6d8b7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.171150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ccb05-9c04-4df1-a3b8-e950c6d8b7b4 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.173204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 472/961] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:31.179542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7464-1c29-454f-a13c-ed7373b7b54c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.180222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7464-1c29-454f-a13c-ed7373b7b54c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.180733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d7464-1c29-454f-a13c-ed7373b7b54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.200033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1f0d7464-1c29-454f-a13c-ed7373b7b54c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:31.201982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1f0d7464-1c29-454f-a13c-ed7373b7b54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:31.277499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:31.287523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 490/962] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:31.307715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ccb05-9c04-4df1-a3b8-e950c6d8b7b4 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:31.418012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ccb05-9c04-4df1-a3b8-e950c6d8b7b4 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:31.450068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf90d4-c50c-4630-88c0-f261cf192c74 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.450391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf90d4-c50c-4630-88c0-f261cf192c74 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.450642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf90d4-c50c-4630-88c0-f261cf192c74 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.452422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 491/963] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:31.457907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44e995-7f87-406e-8f2c-74d6decc4066 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.458285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44e995-7f87-406e-8f2c-74d6decc4066 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.458602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de44e995-7f87-406e-8f2c-74d6decc4066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.458706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:31.459489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.459804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.460085 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.471619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:31.476238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-de44e995-7f87-406e-8f2c-74d6decc4066 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:31.477443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-de44e995-7f87-406e-8f2c-74d6decc4066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:31.480671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:31.481920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:31.482803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:31.486853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 473/964] 10.222.0.42 () {56 vars in 1120 bytes} [Thu May 26 21:08:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:31.488538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:31.516149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:31.559850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 492/965] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:31.578699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44e995-7f87-406e-8f2c-74d6decc4066 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:31.597353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a02aa-6936-48e5-9a8a-9c2b71916c0d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:31.634858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:31.635538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.635853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.636132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.636884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:31.637255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:31.638719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:31.652926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 474/966] 10.222.0.42 () {56 vars in 1115 bytes} [Thu May 26 21:08:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:31.654240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:31.692739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de44e995-7f87-406e-8f2c-74d6decc4066 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:31.739514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:31.749669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:31.750682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:31.751562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.752026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.752454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.753809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:31.755922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:31.765791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534c5a0-e2d1-4ad8-8e6b-322204e78a2e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:31.799415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:31.800344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.800698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.801044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.811634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:31.813140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:31.817832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.sql.core [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 21:08:31.817832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 21:08:31.817832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [parameters: {'user_id': '363d91a444024904a574b6d54613ac32', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 21:08:31.817832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80756) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result = self._query(query) May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application conn.query(q) May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result.read() May 26 21:08:31.828432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 21:08:31.829578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application next(self.gen) May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application yield resource May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application next(self.gen) May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.session.rollback() May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.force_reraise() May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise self.value May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application session.commit() May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 21:08:31.830572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._prepare_impl() May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.session.flush() May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._flush(objects) May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application compat.raise_( May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application flush_context.execute() May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rec.execute(self) May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 21:08:31.831494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 21:08:31.833027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result = self._query(query) May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application conn.query(q) May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application result.read() May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 21:08:31.833884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [parameters: {'user_id': '363d91a444024904a574b6d54613ac32', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:31.836249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 21:08:31.838257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 21:08:31.840642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 21:08:31.840642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 21:08:31.840642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 21:08:31.840642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 21:08:31.840642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 21:08:31.840642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:31.841912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:31.845751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:31.854547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:31.854547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:31.854547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:31.854547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:31.854547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 475/967] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:31.854547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:31.856378 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 493/968] 10.222.0.42 () {60 vars in 1131 bytes} [Thu May 26 21:08:31 2022] POST /identity/v3/users => generated 160 bytes in 276 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:31.856632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:31.953332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad82b23-9ed2-430d-ac46-22c9c6cd0c0c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:31.991822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:31.992769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:31.993126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3f94-22ca-49e3-8812-385fe35adf4e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:31.993385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:31.993833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:31.994707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:31.995171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:31.996740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.005726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:32.016124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 476/969] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:32.018216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.037514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:32.038639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.038973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.039259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.050267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:32.052592 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:32.071562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:32.073156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:32.079195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 494/970] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:32.081723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.134868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993389d5-83a9-4185-a4ae-9100c032c1bb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:32.154872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:32.172598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:32.173809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.174315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.174792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.185915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:32.187717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:32.204353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:32.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 21:08:32.206293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:32.213815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d227c1-3ffe-433f-bfd3-32710f00e4eb None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:32.215254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 477/971] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:32.216489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.255142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:32.256076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.256662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.257142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.258303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:32.258826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:32.261090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.284605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:32.305619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 495/972] 10.222.0.42 () {56 vars in 1101 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:32.335439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337db5b-b125-42fe-9d50-b68b9f7a0af3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:32.377407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb698215-cf24-46af-9883-4efab6d283f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:32.378118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb698215-cf24-46af-9883-4efab6d283f3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.378408 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb698215-cf24-46af-9883-4efab6d283f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.378689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb698215-cf24-46af-9883-4efab6d283f3 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.379555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb698215-cf24-46af-9883-4efab6d283f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:32.379895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb698215-cf24-46af-9883-4efab6d283f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:32.381384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb698215-cf24-46af-9883-4efab6d283f3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.402749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 478/973] 10.222.0.42 () {56 vars in 1149 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:32.408589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.526296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13cf37-72f6-411d-9cf9-126dbeaed6c4 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:32.531851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba83d3-ea9d-4b3b-a384-fc7b126cead0 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.532347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba83d3-ea9d-4b3b-a384-fc7b126cead0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.532347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba83d3-ea9d-4b3b-a384-fc7b126cead0 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.533867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 479/974] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:32.541347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:32.542168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cb2e9-15d6-4b24-9760-57f7cf7ad259 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.542355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cb2e9-15d6-4b24-9760-57f7cf7ad259 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.542614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cb2e9-15d6-4b24-9760-57f7cf7ad259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.560456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-4d0cb2e9-15d6-4b24-9760-57f7cf7ad259 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:32.561715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4d0cb2e9-15d6-4b24-9760-57f7cf7ad259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:32.571040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:32.572526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.572658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.572866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.656906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 480/975] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:32.665831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:32.668106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.688588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cb2e9-15d6-4b24-9760-57f7cf7ad259 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.739927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:32.781530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 496/976] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:32.787259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.800532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:32.803844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cb2e9-15d6-4b24-9760-57f7cf7ad259 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:32.845012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:32.845825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.846111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.846388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.847042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:32.848446 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:32.861193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:32.862241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:32.863090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 481/977] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:32.869126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.902206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299d668-507d-45a4-ae0c-f88d0336ccce None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:32.939701 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:32.940524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:32.940811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:32.941092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:32.952857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:32.954330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:32.974359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:32.975310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:32.976427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:32.976427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:32.976427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:32.976427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:32.976427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 497/978] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:32.981019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:32.985628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc03e3-3a97-4fdb-8a0a-6372eb79605e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:32.998245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:33.021711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.022412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.022896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.023323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.024492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:33.025433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.026787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.036639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 482/979] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:33.037572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:33.051115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:33.092195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119f63e-287d-4877-b0c9-245062386685 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.132569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.133234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.133515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.133789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.134543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:33.134846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.136298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.146573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6501f-0e4f-4dd4-bb7f-3cfc766a6c0a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.147998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 498/980] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:33.153656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:33.184821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.185503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.185774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.186048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.186754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:33.187082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.188457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.198415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 483/981] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:33.200439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:33.248450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:33.269937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97f0e03-64b4-4bd7-bec1-00d2412e281e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.293919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:33.307065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.308165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.308512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.308787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.318473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.318818 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4b757-edbc-4e00-a489-5834c07226d1 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.320223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:33.341341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:33.342321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:33.343218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:33.347942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 499/982] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:33.348610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:33.355850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.356509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.356790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.357061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.357857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] RBAC: Adding query filter params (interface=admin, service_id=10430a895e5642d28a55000cb7674950, region_id=RegionOne) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:33.358173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.359534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.369962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 484/983] 10.222.0.42 () {56 vars in 1249 bytes} [Thu May 26 21:08:33 2022] GET /identity/v3/endpoints?service_id=10430a895e5642d28a55000cb7674950&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:33.371117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:33.459759 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f85faf-40bd-47c4-ae09-d11541db1bce None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.472911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb0551-a7be-4072-81db-cda1d135eeef None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.474518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:33.495958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24bcd46-2691-4f66-bed5-f69417ad575c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.496689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bcd46-2691-4f66-bed5-f69417ad575c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.496972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bcd46-2691-4f66-bed5-f69417ad575c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.497246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24bcd46-2691-4f66-bed5-f69417ad575c None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.498038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bcd46-2691-4f66-bed5-f69417ad575c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:33.498345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bcd46-2691-4f66-bed5-f69417ad575c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.499811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24bcd46-2691-4f66-bed5-f69417ad575c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.511134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc234b-5f08-4bb9-a4e5-e22f46d7e9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.511829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc234b-5f08-4bb9-a4e5-e22f46d7e9cc None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.516805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 500/984] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:33.516903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc234b-5f08-4bb9-a4e5-e22f46d7e9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.516903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc234b-5f08-4bb9-a4e5-e22f46d7e9cc None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.516903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc234b-5f08-4bb9-a4e5-e22f46d7e9cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.517092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720bb4-40e5-4ef3-b0dc-b75b1d619a1c None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.517092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720bb4-40e5-4ef3-b0dc-b75b1d619a1c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.517092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98720bb4-40e5-4ef3-b0dc-b75b1d619a1c None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.517288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc234b-5f08-4bb9-a4e5-e22f46d7e9cc None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.517356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 501/985] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:33.520492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98720bb4-40e5-4ef3-b0dc-b75b1d619a1c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:33.520948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 485/986] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:33 2022] GET /identity/v3/services => generated 1497 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:08:33.522115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275b8ff-1304-458c-a829-a5f213e24c4b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.522407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275b8ff-1304-458c-a829-a5f213e24c4b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.522689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5275b8ff-1304-458c-a829-a5f213e24c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.541860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5275b8ff-1304-458c-a829-a5f213e24c4b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:33.543240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5275b8ff-1304-458c-a829-a5f213e24c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:33.590466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:33.628334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 486/987] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:33.652230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275b8ff-1304-458c-a829-a5f213e24c4b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:33.660794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98720bb4-40e5-4ef3-b0dc-b75b1d619a1c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.704744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f219f-31f5-4cef-811d-145f5902e355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.705435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f219f-31f5-4cef-811d-145f5902e355 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.705728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f219f-31f5-4cef-811d-145f5902e355 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.705994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f219f-31f5-4cef-811d-145f5902e355 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.707093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f219f-31f5-4cef-811d-145f5902e355 None admin] RBAC: Adding query filter params (role.id=13a9265b6efa4d9da45dade8430e7d87, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=67316e32d66342d58684c1ee3e5894c1) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:33.707410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f219f-31f5-4cef-811d-145f5902e355 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.709033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f219f-31f5-4cef-811d-145f5902e355 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.717395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 502/988] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:33 2022] GET /identity/v3/role_assignments?user.id=67316e32d66342d58684c1ee3e5894c1&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=13a9265b6efa4d9da45dade8430e7d87 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:33.771155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:33.778323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5275b8ff-1304-458c-a829-a5f213e24c4b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:33.818121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce96f07-4fe8-4797-9b91-eff9832d74ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:33.818820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96f07-4fe8-4797-9b91-eff9832d74ce None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:33.819119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96f07-4fe8-4797-9b91-eff9832d74ce None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:33.819379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce96f07-4fe8-4797-9b91-eff9832d74ce None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:33.820431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce96f07-4fe8-4797-9b91-eff9832d74ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:33.822383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce96f07-4fe8-4797-9b91-eff9832d74ce None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:33.898849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 487/989] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:33 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:08:35.786492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b0c96-1991-4dd2-a861-e1a06f2022b7 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:35.787035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b0c96-1991-4dd2-a861-e1a06f2022b7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:35.787035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294b0c96-1991-4dd2-a861-e1a06f2022b7 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:35.788653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 503/990] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:35.795127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4f05e-4765-4445-b23b-2afc6b48e375 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:35.795681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4f05e-4765-4445-b23b-2afc6b48e375 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:35.795681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4f05e-4765-4445-b23b-2afc6b48e375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:35.813813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-0df4f05e-4765-4445-b23b-2afc6b48e375 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:35.815029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0df4f05e-4765-4445-b23b-2afc6b48e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:35.891709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 488/991] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:35.913629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b0c96-1991-4dd2-a861-e1a06f2022b7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:35.991549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b7040-f1dc-44c2-afa2-7bc43e216dd7 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:35.991765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b7040-f1dc-44c2-afa2-7bc43e216dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:35.992065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b7040-f1dc-44c2-afa2-7bc43e216dd7 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:35.993456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 489/992] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:35.998910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b323-b004-4474-9a97-404529f11fbb None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:35.999201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b323-b004-4474-9a97-404529f11fbb None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:35.999484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b323-b004-4474-9a97-404529f11fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.016643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b8f7b323-b004-4474-9a97-404529f11fbb None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:36.017841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b8f7b323-b004-4474-9a97-404529f11fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:36.028611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294b0c96-1991-4dd2-a861-e1a06f2022b7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.035722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:36.064350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.065159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.065443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.065715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.066732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.068474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:36.081857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:36.083103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 21:08:36.084198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:36.086747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 504/993] 10.222.0.42 () {56 vars in 1126 bytes} [Thu May 26 21:08:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:36.093469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.102231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 490/994] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:35 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:36.123866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b323-b004-4474-9a97-404529f11fbb None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.138180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:36.196616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0241f8-5209-4e09-aeb8-49cf307f6eb9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.224741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f7b323-b004-4474-9a97-404529f11fbb None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.229875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a4a64-fccc-4a27-a17e-1d5c90e75e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.230534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a4a64-fccc-4a27-a17e-1d5c90e75e1f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.230814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a4a64-fccc-4a27-a17e-1d5c90e75e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.231083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a4a64-fccc-4a27-a17e-1d5c90e75e1f None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.231809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a4a64-fccc-4a27-a17e-1d5c90e75e1f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:36.232122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a4a64-fccc-4a27-a17e-1d5c90e75e1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.233489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a4a64-fccc-4a27-a17e-1d5c90e75e1f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.238294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 505/995] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:36.239483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65da100-93d8-4fde-b7a5-c81c076048e0 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.239790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65da100-93d8-4fde-b7a5-c81c076048e0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.240068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65da100-93d8-4fde-b7a5-c81c076048e0 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.241281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 506/996] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:36.244366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65da100-93d8-4fde-b7a5-c81c076048e0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.264480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76451e26-8183-4539-88b7-f7ec07b171d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.265276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76451e26-8183-4539-88b7-f7ec07b171d5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.265571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76451e26-8183-4539-88b7-f7ec07b171d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.265840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76451e26-8183-4539-88b7-f7ec07b171d5 None admin] PATH_INFO: `/v3/services/image` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.266494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76451e26-8183-4539-88b7-f7ec07b171d5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.269213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76451e26-8183-4539-88b7-f7ec07b171d5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-76451e26-8183-4539-88b7-f7ec07b171d5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:36.280000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:36.281062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:36.282485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 491/997] 10.222.0.42 () {56 vars in 1105 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:36.283289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b784-9d9a-48f3-b144-74c44a5fbd90 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.283289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b784-9d9a-48f3-b144-74c44a5fbd90 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.283419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b784-9d9a-48f3-b144-74c44a5fbd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.286108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:36.300036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5941b784-9d9a-48f3-b144-74c44a5fbd90 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:36.301210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5941b784-9d9a-48f3-b144-74c44a5fbd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:36.348710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65da100-93d8-4fde-b7a5-c81c076048e0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.369353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 492/998] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:36.370002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941b784-9d9a-48f3-b144-74c44a5fbd90 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.379350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:36.388939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.392630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.392630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.392630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.392630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:36.392630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.392630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.402546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 507/999] 10.222.0.42 () {56 vars in 1122 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:36.402546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.470414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941b784-9d9a-48f3-b144-74c44a5fbd90 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.496677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:36.500007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e43852-4d0a-44c5-af37-e4403895840c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.505000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.507031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.507031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.507031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.507031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:36.507365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.508748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.513806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 493/1000] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:36.514557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.538112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.538906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.539187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.539454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.540207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.542183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:36.554019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:36.555057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:36.556021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 508/1001] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:36.557417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.587029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:36.618014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d7242-550d-4730-bf38-f9c3057537e3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.653710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.654364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.654643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.654916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.655664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.657036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.660042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb6525-15d6-4e09-967b-dbb60eca4734 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.693841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.694506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.695041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.695308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.696037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] RBAC: Adding query filter params (type=image) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:36.696437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.698367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.702921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 509/1002] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:36.703698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:36.734957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:36.815946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 494/1003] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:36 2022] POST /identity/v3/endpoints => generated 325 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:36.828538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381fa1ac-9603-426f-9fd6-c96dc96433ad None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:36.877113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:36.877803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:36.878087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:36.878362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:36.879100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:36.879410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:36.880936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:36.891947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 510/1004] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:36.893500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:37.009121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830d8db9-79ec-43c4-8caa-c854a7c101ac None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:37.038970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:37.044847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:37.045521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:37.045800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:37.051953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:37.051953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:37.051953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:37.051953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:37.053350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 495/1005] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:37.060337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:37.153146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:37.168614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c69929-21fc-46d6-b362-4569eb6b4073 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:37.205312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a70ce83-a415-4ced-aca5-8042c50c34d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:37.205979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70ce83-a415-4ced-aca5-8042c50c34d9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:37.206263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70ce83-a415-4ced-aca5-8042c50c34d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:37.206548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a70ce83-a415-4ced-aca5-8042c50c34d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:37.207336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70ce83-a415-4ced-aca5-8042c50c34d9 None admin] RBAC: Adding query filter params (interface=public, service_id=9217f2aaf9604ba0b162bed874387b40, region_id=RegionOne) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:37.207650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70ce83-a415-4ced-aca5-8042c50c34d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:37.209087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a70ce83-a415-4ced-aca5-8042c50c34d9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:37.219408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 511/1006] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:08:37 2022] GET /identity/v3/endpoints?service_id=9217f2aaf9604ba0b162bed874387b40&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:37.220429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:37.326330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5143071-0119-46c2-a2dd-c9971a70e8b5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:37.361600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a472f-119a-4c2c-9589-28321c41e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:37.362269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a472f-119a-4c2c-9589-28321c41e288 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:37.364276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a472f-119a-4c2c-9589-28321c41e288 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:37.364276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a472f-119a-4c2c-9589-28321c41e288 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:37.364276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a472f-119a-4c2c-9589-28321c41e288 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:37.364954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a472f-119a-4c2c-9589-28321c41e288 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:37.371646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 496/1007] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:37 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:08:38.963069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77d877-7db1-4af7-bb0b-05a9ddec3a37 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:38.963575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77d877-7db1-4af7-bb0b-05a9ddec3a37 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:38.963575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b77d877-7db1-4af7-bb0b-05a9ddec3a37 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:38.965215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 512/1008] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:38.970564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5e546-9ac3-4eae-95de-9d487bbb39a9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:38.970957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5e546-9ac3-4eae-95de-9d487bbb39a9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:38.971133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5e546-9ac3-4eae-95de-9d487bbb39a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:38.989071 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-49c5e546-9ac3-4eae-95de-9d487bbb39a9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:38.990285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-49c5e546-9ac3-4eae-95de-9d487bbb39a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:39.046989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:39.063932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 497/1009] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:38 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:39.088015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77d877-7db1-4af7-bb0b-05a9ddec3a37 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:39.199231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b77d877-7db1-4af7-bb0b-05a9ddec3a37 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:39.233947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:39.234532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.234819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.235188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.235927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:39.237311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:39.255408 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203ca74-246c-45bb-8da5-e3726bfee4dc None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.255693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203ca74-246c-45bb-8da5-e3726bfee4dc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.256005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203ca74-246c-45bb-8da5-e3726bfee4dc None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.257489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 498/1010] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:39.263107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c20c1-761f-450e-9220-6376db626ee3 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.263394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c20c1-761f-450e-9220-6376db626ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.263670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c20c1-761f-450e-9220-6376db626ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.280689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-936c20c1-761f-450e-9220-6376db626ee3 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:39.281882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-936c20c1-761f-450e-9220-6376db626ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:39.349076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 499/1011] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:39 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:39.375795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 513/1012] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:39 2022] POST /identity/v3/services => generated 242 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:39.377069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c20c1-761f-450e-9220-6376db626ee3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:39.495286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c20c1-761f-450e-9220-6376db626ee3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:39.514985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:39.541239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d91f5d-dbcc-4cef-92f8-88933888496d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:39.542055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d91f5d-dbcc-4cef-92f8-88933888496d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.542334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d91f5d-dbcc-4cef-92f8-88933888496d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.542605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d91f5d-dbcc-4cef-92f8-88933888496d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.543844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d91f5d-dbcc-4cef-92f8-88933888496d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:39.545299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d91f5d-dbcc-4cef-92f8-88933888496d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:39.547184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 500/1013] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:39.552770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:39.566908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:39.618034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945249-3591-4886-a58b-6455f818ac51 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.618344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945249-3591-4886-a58b-6455f818ac51 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.618640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90945249-3591-4886-a58b-6455f818ac51 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.620193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 501/1014] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:39.625412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b2d8-7280-407d-aa66-550592f897f4 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.625735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b2d8-7280-407d-aa66-550592f897f4 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.626077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1b2d8-7280-407d-aa66-550592f897f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.643300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b0d1b2d8-7280-407d-aa66-550592f897f4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:39.644529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b0d1b2d8-7280-407d-aa66-550592f897f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:39.663786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbb65a-8cb7-4b3d-8534-3a5518601288 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:39.708197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:39.708872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.709161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.709439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.710167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:39.711613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:39.721854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 502/1015] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:39 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:39.744740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1b2d8-7280-407d-aa66-550592f897f4 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:39.843066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1b2d8-7280-407d-aa66-550592f897f4 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:39.858013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:39.860464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 514/1016] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:08:39 2022] POST /identity/v3/projects => generated 311 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:08:39.891099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:39.891963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:39.892246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:39.892555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:39.893226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:39.894774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:39.908659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:39.911701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:39.912802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 503/1017] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:39.916848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:39.988114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:40.029149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab25aa1-482c-4548-8469-8fd412678e67 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:40.066460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:40.068976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:40.068976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:40.068976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:40.068976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:40.068976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:40.070532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:40.076482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 515/1018] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:40.082285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:40.198349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb179b44-7e34-4915-8950-d58b4586e32a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:40.233590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca15dcf-6fbb-443f-a1d1-c6fbe4116566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:40.234259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca15dcf-6fbb-443f-a1d1-c6fbe4116566 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:40.234543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca15dcf-6fbb-443f-a1d1-c6fbe4116566 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:40.234819 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca15dcf-6fbb-443f-a1d1-c6fbe4116566 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:40.235531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca15dcf-6fbb-443f-a1d1-c6fbe4116566 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:40.235866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca15dcf-6fbb-443f-a1d1-c6fbe4116566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:40.237242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca15dcf-6fbb-443f-a1d1-c6fbe4116566 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:40.242397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 504/1019] 10.222.0.42 () {56 vars in 1112 bytes} [Thu May 26 21:08:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:40.248222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:40.352102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1828ad6-7db7-4cf6-92b9-8b6f7aa2470a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:40.377315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:40.387023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446ab1f-8971-46cc-9a6a-ac9621d3fad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:40.387689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ab1f-8971-46cc-9a6a-ac9621d3fad4 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:40.388013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ab1f-8971-46cc-9a6a-ac9621d3fad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:40.388336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446ab1f-8971-46cc-9a6a-ac9621d3fad4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:40.397859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446ab1f-8971-46cc-9a6a-ac9621d3fad4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:40.397859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446ab1f-8971-46cc-9a6a-ac9621d3fad4 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:40.504278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 516/1020] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:41.650558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164028-2037-48c4-9075-5a35c63652b3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:41.651116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164028-2037-48c4-9075-5a35c63652b3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:41.651116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28164028-2037-48c4-9075-5a35c63652b3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:41.652561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 505/1021] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:41.657990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfb36-b643-496d-bdca-64830eabfbfe None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:41.658428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfb36-b643-496d-bdca-64830eabfbfe None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:41.658614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cfb36-b643-496d-bdca-64830eabfbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:41.675948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-710cfb36-b643-496d-bdca-64830eabfbfe None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:41.677469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-710cfb36-b643-496d-bdca-64830eabfbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:41.761745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 517/1022] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:41.784602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28164028-2037-48c4-9075-5a35c63652b3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:41.830564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:41.892271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28164028-2037-48c4-9075-5a35c63652b3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:41.926786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:41.927587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:41.927904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:41.928181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] PATH_INFO: `/v3/services/image` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:41.928832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:41.930196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:41.942618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:41.943674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:41.950534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 506/1023] 10.222.0.42 () {56 vars in 1105 bytes} [Thu May 26 21:08:41 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:41.951707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfb36-b643-496d-bdca-64830eabfbfe None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:42.059426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cfb36-b643-496d-bdca-64830eabfbfe None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:42.068764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:42.094099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da959a0-8333-4ca6-8282-950ebba1844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:42.094764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da959a0-8333-4ca6-8282-950ebba1844b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.095044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da959a0-8333-4ca6-8282-950ebba1844b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.095317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da959a0-8333-4ca6-8282-950ebba1844b None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.096048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da959a0-8333-4ca6-8282-950ebba1844b None admin] RBAC: Adding query filter params (name=image) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:42.096357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da959a0-8333-4ca6-8282-950ebba1844b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:42.097727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da959a0-8333-4ca6-8282-950ebba1844b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:42.107887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 518/1024] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:42.108379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:42.153919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd71b0-0824-4760-9c72-0f4c823a44ee None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.154810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd71b0-0824-4760-9c72-0f4c823a44ee None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.155091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd71b0-0824-4760-9c72-0f4c823a44ee None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.156807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 519/1025] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:42.162041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278b82d-3a2f-4265-b8fa-b54d50db4532 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.162328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278b82d-3a2f-4265-b8fa-b54d50db4532 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.162606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278b82d-3a2f-4265-b8fa-b54d50db4532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.179155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f278b82d-3a2f-4265-b8fa-b54d50db4532 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:42.180395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f278b82d-3a2f-4265-b8fa-b54d50db4532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:42.218133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff269b0-88e2-4a1a-8206-dfb973c0e097 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:42.220657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:42.248660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 520/1026] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:42.254559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:42.255223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.255504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.255806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.256550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] RBAC: Adding query filter params (type=image) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:42.256878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:42.258193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:42.268079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 507/1027] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:42 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:42.269842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278b82d-3a2f-4265-b8fa-b54d50db4532 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:42.273160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:42.380138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116bb7ea-6b55-4a4e-9788-a2d6d39938eb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:42.381323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278b82d-3a2f-4265-b8fa-b54d50db4532 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:42.383224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:42.418068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:42.418307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:42.418876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.419002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.419213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.419493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.420067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.420349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.420887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:42.421146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] RBAC: Adding query filter params (interface=public, service_id=112c146bc91a4cef812d70055026b3fa, region_id=RegionOne) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:42.421464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:42.422756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:42.423338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:42.424166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 508/1028] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:42.433522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 521/1029] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:08:42 2022] GET /identity/v3/endpoints?service_id=112c146bc91a4cef812d70055026b3fa&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:42.434840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:42.434976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:42.474047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:42.542051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c384a-72e9-4dd7-9453-883cc39797a0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:42.544893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad85349-fd78-48bc-8e33-a61fa0d090e8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:42.580152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a35c5-6169-43ef-8584-2bfc2e34981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:42.580827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a35c5-6169-43ef-8584-2bfc2e34981c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.581120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a35c5-6169-43ef-8584-2bfc2e34981c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.581401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a35c5-6169-43ef-8584-2bfc2e34981c None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.582116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a35c5-6169-43ef-8584-2bfc2e34981c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:42.582300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361ccb2-e8a4-471f-969f-c740249155df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:42.582929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ccb2-e8a4-471f-969f-c740249155df None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.583212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ccb2-e8a4-471f-969f-c740249155df None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.583486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361ccb2-e8a4-471f-969f-c740249155df None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.583594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a35c5-6169-43ef-8584-2bfc2e34981c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:42.584226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361ccb2-e8a4-471f-969f-c740249155df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:42.585639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361ccb2-e8a4-471f-969f-c740249155df None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:42.591183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 522/1030] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:08:42 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:08:42.719134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 509/1031] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:08:42 2022] POST /identity/v3/users => generated 285 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:42.830906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380fdb7-4003-412e-9a43-ec13fa80ab61 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.831468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380fdb7-4003-412e-9a43-ec13fa80ab61 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.831468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380fdb7-4003-412e-9a43-ec13fa80ab61 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.833154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 523/1032] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:42.838991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761aae-c954-4d08-ae51-4f112d0d5288 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:42.839291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761aae-c954-4d08-ae51-4f112d0d5288 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:42.839713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70761aae-c954-4d08-ae51-4f112d0d5288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:42.859011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-70761aae-c954-4d08-ae51-4f112d0d5288 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:42.860247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-70761aae-c954-4d08-ae51-4f112d0d5288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:42.935294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 510/1033] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:42.959371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380fdb7-4003-412e-9a43-ec13fa80ab61 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:42.994062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:43.071076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380fdb7-4003-412e-9a43-ec13fa80ab61 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:43.108322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:43.109140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:43.109423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:43.109695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:43.120674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:43.128833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:43.129779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:43.130656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 524/1034] 10.222.0.42 () {56 vars in 1120 bytes} [Thu May 26 21:08:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:43.136340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70761aae-c954-4d08-ae51-4f112d0d5288 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:43.161891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:43.238792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70761aae-c954-4d08-ae51-4f112d0d5288 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:43.275001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:43.275673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:43.275984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:43.276258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:43.277010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:43.277317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:43.278687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:43.287312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 511/1035] 10.222.0.42 () {56 vars in 1115 bytes} [Thu May 26 21:08:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:43.293303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:43.388479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:43.398658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77fc93-e8fd-4b18-be25-37e2426dab6c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:43.434168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:43.434968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:43.435253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:43.435531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:43.447201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:43.448828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:43.468220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:43.469217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:43.475117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:43.475117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:43.475117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:43.475117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:43.475117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 525/1036] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:43.475529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:43.565863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:43.583789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0aea3-daf0-4209-9f10-bd7dd69fdc66 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:43.619201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:43.619907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:43.620196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:43.620476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:43.621204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:43.621516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:43.622936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:43.636526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 512/1037] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:43.642130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:43.755065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47004c7d-8b8a-4446-b485-ade02f16c138 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:43.791710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:43.792541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:43.792826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:43.793104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:43.800989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:43.802516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:43.818237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:43.824997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:43.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 526/1038] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:08:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:43.826565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:43.931026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6146e87f-0fb1-4465-9fb8-f449edb7a439 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:43.965833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:43.966509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:43.966793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:43.967070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:43.968032 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:43.968261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:43.969688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:43.993207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 513/1039] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:08:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:43.995168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:43.998232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:44.098821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9aec13-f843-4aa5-bc8f-66e81303a064 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:44.133298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:44.133976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:44.134259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:44.134534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:44.201212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:44.202673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:44.204592 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:44.313072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 527/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:43 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 21:08:44.314459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:44.317642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:44.417715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf5c7f-ab6d-42d9-bb33-bf2b92b31643 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:44.456818 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:44.457877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:44.458329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:44.458762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:44.470156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:44.471933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:44.490361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:44.491512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:44.492460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:44.492460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:44.492460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:44.492460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:44.492460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 514/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:44.499233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:44.602586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24bdb8f-32d5-4cc8-8589-c52911d348da None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:44.604194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:44.638267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819da9a-78b7-4b4d-a73c-177d6811be86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:44.638943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819da9a-78b7-4b4d-a73c-177d6811be86 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:44.639327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819da9a-78b7-4b4d-a73c-177d6811be86 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:44.639561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819da9a-78b7-4b4d-a73c-177d6811be86 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:44.640336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819da9a-78b7-4b4d-a73c-177d6811be86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:44.640647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819da9a-78b7-4b4d-a73c-177d6811be86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:44.642060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819da9a-78b7-4b4d-a73c-177d6811be86 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:44.657353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 528/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:44.662857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:44.733373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:44.779862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149a0e-16ac-4891-90ab-0af7e6c1e65c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:44.816556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:44.817639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:44.818068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:44.818505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:44.834158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:44.835634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:44.850175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc488069-1cb5-47c5-bfa8-d368d016c824 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:44.850480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc488069-1cb5-47c5-bfa8-d368d016c824 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:44.850749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc488069-1cb5-47c5-bfa8-d368d016c824 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:44.851204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:44.852169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:44.856826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:44.857282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 529/1043] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:44.857361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 515/1044] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:44.857438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771056-a98c-4f51-a03f-9c9a4b500d91 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:44.859075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771056-a98c-4f51-a03f-9c9a4b500d91 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:44.859075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77771056-a98c-4f51-a03f-9c9a4b500d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:44.859915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:44.875845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-77771056-a98c-4f51-a03f-9c9a4b500d91 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:44.877081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-77771056-a98c-4f51-a03f-9c9a4b500d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:44.952505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 530/1045] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:44.974384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af9884-aa84-4cce-909a-e2beb334ef56 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:44.975364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771056-a98c-4f51-a03f-9c9a4b500d91 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:44.992308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:45.013879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd4d07b-4e18-486f-b265-8788b4427ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.014550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d07b-4e18-486f-b265-8788b4427ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.014838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d07b-4e18-486f-b265-8788b4427ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.015792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd4d07b-4e18-486f-b265-8788b4427ad6 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.015981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4d07b-4e18-486f-b265-8788b4427ad6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:45.016293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4d07b-4e18-486f-b265-8788b4427ad6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.017724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd4d07b-4e18-486f-b265-8788b4427ad6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.029727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 516/1046] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:45.031029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc84d3-3a01-4cb1-afdc-53440af50e18 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.031490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc84d3-3a01-4cb1-afdc-53440af50e18 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.031688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc84d3-3a01-4cb1-afdc-53440af50e18 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.035149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 517/1047] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:45.037228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc84d3-3a01-4cb1-afdc-53440af50e18 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.076497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:45.084608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77771056-a98c-4f51-a03f-9c9a4b500d91 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.117611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c07fb52-ed96-455a-b3e1-3df7f1c6957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.118999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07fb52-ed96-455a-b3e1-3df7f1c6957c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.119283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07fb52-ed96-455a-b3e1-3df7f1c6957c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.119558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c07fb52-ed96-455a-b3e1-3df7f1c6957c None admin] PATH_INFO: `/v3/services/image` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.120241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07fb52-ed96-455a-b3e1-3df7f1c6957c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.121605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c07fb52-ed96-455a-b3e1-3df7f1c6957c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-8c07fb52-ed96-455a-b3e1-3df7f1c6957c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:45.132904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:45.133903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:45.134990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 531/1048] 10.222.0.42 () {56 vars in 1105 bytes} [Thu May 26 21:08:44 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:45.135994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a364ce0-86d8-4bf1-a19c-3a5eb1a68b2e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.135994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a364ce0-86d8-4bf1-a19c-3a5eb1a68b2e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.135994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a364ce0-86d8-4bf1-a19c-3a5eb1a68b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.140531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc84d3-3a01-4cb1-afdc-53440af50e18 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.152650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1a364ce0-86d8-4bf1-a19c-3a5eb1a68b2e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:45.153840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1a364ce0-86d8-4bf1-a19c-3a5eb1a68b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:45.176085 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.176743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.177026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.177299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.178128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] RBAC: Adding query filter params (role.id=13a9265b6efa4d9da45dade8430e7d87, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=eefd00457d354271867932d336d7f69c) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:45.178443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.179880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.185275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 518/1049] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:45 2022] GET /identity/v3/role_assignments?user.id=eefd00457d354271867932d336d7f69c&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=13a9265b6efa4d9da45dade8430e7d87 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:45.186119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.235652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:45.242586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 532/1050] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:45.267511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a364ce0-86d8-4bf1-a19c-3a5eb1a68b2e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.305406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e3475-1697-4812-8804-dfcf0e0d90f6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.345075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.345763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.346044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.346318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.347040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] RBAC: Adding query filter params (name=image) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:45.347491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.349067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.354686 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 519/1051] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:45 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:45.361137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.376375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:45.393447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a364ce0-86d8-4bf1-a19c-3a5eb1a68b2e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.429980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.431434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.431714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.432015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.442952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:45.452025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:45.453003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:45.453865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 533/1052] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:45.459232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.467873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e14557f-6ed4-4a7f-8dbe-af30605ccf7e None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.469466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:45.509691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.510633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.511112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.511643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.512667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] RBAC: Adding query filter params (type=image) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:45.513212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.514758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.521095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 520/1053] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:45 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:45.528634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.574383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6324d18-c965-432d-b5b3-c888580f2400 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.612526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.613194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.613480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.613738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.614478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:45.614782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.616181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.629164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 534/1054] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:45.630331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.637627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac5aaf0-fd7e-4bfc-a6c0-40c883306c96 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.642021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:45.675737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.676590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.676882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.677151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.677908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.679287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.738724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bbbae8-2101-4e88-8d81-131b6a55b35b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.771959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.772764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.773048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.773320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] PATH_INFO: `/v3/users/ec0d0c3383814a1192b653b1fbdf1042` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:45.787104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] RBAC: Authorizing `identity:get_user(user_id=ec0d0c3383814a1192b653b1fbdf1042)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:45.788583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:45.795058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 535/1055] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:08:45 2022] GET /identity/v3/users/ec0d0c3383814a1192b653b1fbdf1042 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:45.801840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 521/1056] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:08:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:08:45.801992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:45.873695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:45.912890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e948-c5d1-4741-9eb9-be2bffa5c53c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:45.958232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:45.958973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:45.959263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:45.959543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:46.035474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:46.036927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:46.099642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:46.140045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 536/1057] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:46.145601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:46.171793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:46.252281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75bac3-751b-4a96-b80c-87ca3f4aa46c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:46.287230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3364b070-6920-4e37-b1bc-520c1f7252c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:46.288073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364b070-6920-4e37-b1bc-520c1f7252c2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:46.288365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364b070-6920-4e37-b1bc-520c1f7252c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:46.288634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3364b070-6920-4e37-b1bc-520c1f7252c2 None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:46.289987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364b070-6920-4e37-b1bc-520c1f7252c2 None admin] RBAC: Authorizing `identity:get_project(project_id=fe379f778baa42ec8af8518892165fad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:46.292052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3364b070-6920-4e37-b1bc-520c1f7252c2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:46.294069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 522/1058] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:08:46 2022] GET /identity/v3/projects/fe379f778baa42ec8af8518892165fad => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:46.300259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:46.404597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d74ca-a49e-469b-b16d-692034a5dcba None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:46.440275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4aa818-2236-4058-9381-51bfee1119fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:46.440931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa818-2236-4058-9381-51bfee1119fd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:46.441215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa818-2236-4058-9381-51bfee1119fd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:46.441491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4aa818-2236-4058-9381-51bfee1119fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:46.442311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa818-2236-4058-9381-51bfee1119fd None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=fe379f778baa42ec8af8518892165fad, user.id=ec0d0c3383814a1192b653b1fbdf1042) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:46.442626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa818-2236-4058-9381-51bfee1119fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:46.444079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4aa818-2236-4058-9381-51bfee1119fd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:46.455891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 537/1059] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:46 2022] GET /identity/v3/role_assignments?user.id=ec0d0c3383814a1192b653b1fbdf1042&scope.project.id=fe379f778baa42ec8af8518892165fad&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:47.391689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775852f-07f8-40e0-bf04-2f3d4de393c6 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:47.392260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775852f-07f8-40e0-bf04-2f3d4de393c6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:47.392260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775852f-07f8-40e0-bf04-2f3d4de393c6 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:47.393654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 523/1060] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:47.398820 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc047-8497-4a38-9de9-453945947b4d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:47.399201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc047-8497-4a38-9de9-453945947b4d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:47.399422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc047-8497-4a38-9de9-453945947b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:47.416516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-959fc047-8497-4a38-9de9-453945947b4d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:47.417748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-959fc047-8497-4a38-9de9-453945947b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:47.439499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:47.488347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 538/1061] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:47.511120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775852f-07f8-40e0-bf04-2f3d4de393c6 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:47.617336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:47.629388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775852f-07f8-40e0-bf04-2f3d4de393c6 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:47.666030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:47.666760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:47.667039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:47.667307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:47.677573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:47.686447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:47.687386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:47.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:47.692200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 524/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Thu May 26 21:08:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:47.694117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc047-8497-4a38-9de9-453945947b4d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:47.804780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc047-8497-4a38-9de9-453945947b4d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:47.839735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:47.840425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:47.840704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:47.840974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:47.841716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:47.842019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:47.843384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:47.856348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 539/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Thu May 26 21:08:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:47.857665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:47.966047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7d0e7-9145-4258-9b62-af54316261d7 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:47.999184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a05676-092b-47ec-9555-5ab05ff07661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.000013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05676-092b-47ec-9555-5ab05ff07661 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.000297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05676-092b-47ec-9555-5ab05ff07661 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.000569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a05676-092b-47ec-9555-5ab05ff07661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.011689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a05676-092b-47ec-9555-5ab05ff07661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.013155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a05676-092b-47ec-9555-5ab05ff07661 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-11a05676-092b-47ec-9555-5ab05ff07661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:48.030711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:48.031680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:48.033274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:48.033274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:48.033274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:48.033274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:48.036111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 525/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:48.037564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.059202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.080329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b3bf-c2e6-4dee-8977-2c4f255409de None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.080609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b3bf-c2e6-4dee-8977-2c4f255409de None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.085598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23b3bf-c2e6-4dee-8977-2c4f255409de None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.085598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 526/1065] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:48.087442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052795f-29a5-460a-b0b4-02d6fc2c1afc None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.087732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052795f-29a5-460a-b0b4-02d6fc2c1afc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.088048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052795f-29a5-460a-b0b4-02d6fc2c1afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.093326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.106456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5052795f-29a5-460a-b0b4-02d6fc2c1afc None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:48.107670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5052795f-29a5-460a-b0b4-02d6fc2c1afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:48.147017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698ab82-89e2-4a7c-b90a-70123f2082c5 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:48.177700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 527/1066] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:48.185703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.186371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.186654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.186927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.187648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:48.187978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.189345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.199757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052795f-29a5-460a-b0b4-02d6fc2c1afc None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.202042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 540/1067] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:48.208572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.291314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.312586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052795f-29a5-460a-b0b4-02d6fc2c1afc None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:48.317885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c3b5-3775-4f76-9764-3da1b1fb55ca None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:48.344504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.350200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.350854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.351136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.356835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.357027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.357027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.357027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.357027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.364590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.366048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:48.383337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:48.384933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:48.385869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 541/1068] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:48.390922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.425927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.427421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.504874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5d928-57e0-41f9-972a-033f58222491 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:48.537374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 528/1069] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:48.541772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.542675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.542782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.543035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.543555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.543958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:48.544277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.545711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.553796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.565325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 542/1070] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:48.571905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.576910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.649967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190ced7-ea56-4423-a9be-71efafed47cd None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:48.677118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cc329-b2ec-4e08-a463-1e6cbc0585da None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:48.686812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc72db9-49d5-4a10-b03d-8dca157408d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.687605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72db9-49d5-4a10-b03d-8dca157408d6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.687918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72db9-49d5-4a10-b03d-8dca157408d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.688193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72db9-49d5-4a10-b03d-8dca157408d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.699922 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc72db9-49d5-4a10-b03d-8dca157408d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.701429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc72db9-49d5-4a10-b03d-8dca157408d6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.716939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ab033-8e1f-4adf-8546-1dfc31219d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.717737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab033-8e1f-4adf-8546-1dfc31219d35 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.718024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab033-8e1f-4adf-8546-1dfc31219d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.718302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ab033-8e1f-4adf-8546-1dfc31219d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-2fc72db9-49d5-4a10-b03d-8dca157408d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:48.719801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:48.721183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:48.722148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:48.722148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:48.722148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:48.722148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:48.722148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 529/1071] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:48.722426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b475fc7-a28f-420b-9000-8cdcbefbb944 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.722551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b475fc7-a28f-420b-9000-8cdcbefbb944 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.722821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b475fc7-a28f-420b-9000-8cdcbefbb944 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.724103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 530/1072] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:48.726757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b475fc7-a28f-420b-9000-8cdcbefbb944 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.733792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ab033-8e1f-4adf-8546-1dfc31219d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.735262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ab033-8e1f-4adf-8546-1dfc31219d35 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-b04ab033-8e1f-4adf-8546-1dfc31219d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:48.753005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:48.754108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:48.755212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:48.755212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:48.755212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:48.755212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:48.755212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 543/1073] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:48.755669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc06a-55f0-4c0d-a040-e7f49934bd13 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.755926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc06a-55f0-4c0d-a040-e7f49934bd13 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.756360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc06a-55f0-4c0d-a040-e7f49934bd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.772929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-69ccc06a-55f0-4c0d-a040-e7f49934bd13 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:48.774131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-69ccc06a-55f0-4c0d-a040-e7f49934bd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:48.814990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.818107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:48.832318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b475fc7-a28f-420b-9000-8cdcbefbb944 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:48.841670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 544/1074] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:48.842590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc06a-55f0-4c0d-a040-e7f49934bd13 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.868673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:48.869571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:48.869854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:48.870129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:48.870843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:48.871150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:48.872622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:48.884288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 531/1075] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:48.885189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:48.972279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc06a-55f0-4c0d-a040-e7f49934bd13 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.003761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08198b79-2dcf-4447-aef9-8b18550e7fbe None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.012026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.012713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.012998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.013276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.014013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:49.014330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.015776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.033651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 545/1076] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:49.035481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:49.049122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.049824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.050109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.050562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.069902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:49.078570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:49.079571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:49.080478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:49.082351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 532/1077] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:49.085638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:49.085853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:49.091465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:49.168445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badfa5f-1ec6-4cb1-83cf-745c1ff574c8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.212990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4f5da-dca2-4a64-9904-c7a802a45e98 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.218558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.219465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.219797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.220053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.227109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.229241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:49.244690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:49.245638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:49.246703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 546/1078] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:49.251638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:49.252627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.253299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.253607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.253884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.254709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:49.254939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.256356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.265581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 533/1079] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:49.271283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:49.338179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:49.350910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:49.355349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79441e17-17ee-4bb7-81be-579e9361387e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.373000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3a4d6-fd0f-4b23-ba4c-6733d74ba2e5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.389706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.389706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.389706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.389706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.390063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:49.390371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.391816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.403006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 547/1080] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:49.404901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.405765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.406047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.406320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] PATH_INFO: `/v3/users/ec0d0c3383814a1192b653b1fbdf1042` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.407516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] RBAC: Authorizing `identity:get_user(user_id=ec0d0c3383814a1192b653b1fbdf1042)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.408875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:49.409255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.411228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 534/1081] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:08:49 2022] GET /identity/v3/users/ec0d0c3383814a1192b653b1fbdf1042 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:49.416933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:49.509863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a5e9ae-5bde-429c-aa78-6f2d3aca97c0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.516381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022419c-9ea8-4f22-8d5a-b387f28b279b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.544059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952e1b5a-3e5e-4b0c-b81d-12b99f9295d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.544985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e1b5a-3e5e-4b0c-b81d-12b99f9295d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.545270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e1b5a-3e5e-4b0c-b81d-12b99f9295d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.545545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952e1b5a-3e5e-4b0c-b81d-12b99f9295d5 None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/eefd00457d354271867932d336d7f69c/roles/13a9265b6efa4d9da45dade8430e7d87` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.547763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e1b5a-3e5e-4b0c-b81d-12b99f9295d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=eefd00457d354271867932d336d7f69c, role_id=13a9265b6efa4d9da45dade8430e7d87)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.549213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952e1b5a-3e5e-4b0c-b81d-12b99f9295d5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.550138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.551026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.551292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.551568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.552796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] RBAC: Authorizing `identity:get_project(project_id=fe379f778baa42ec8af8518892165fad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.554201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.556516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 535/1082] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:08:49 2022] GET /identity/v3/projects/fe379f778baa42ec8af8518892165fad => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:49.562279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:49.564980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:49.617705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 548/1083] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:08:49 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/eefd00457d354271867932d336d7f69c/roles/13a9265b6efa4d9da45dade8430e7d87 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:08:49.676992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be8ace-70f3-4475-9801-0a62408307eb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:49.715610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a47bab-176d-4cd9-8f6e-9fe8e8e4d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:49.716569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47bab-176d-4cd9-8f6e-9fe8e8e4d017 None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:49.716851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47bab-176d-4cd9-8f6e-9fe8e8e4d017 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:49.717132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a47bab-176d-4cd9-8f6e-9fe8e8e4d017 None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad/users/ec0d0c3383814a1192b653b1fbdf1042/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:49.719298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47bab-176d-4cd9-8f6e-9fe8e8e4d017 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe379f778baa42ec8af8518892165fad, user_id=ec0d0c3383814a1192b653b1fbdf1042, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:49.720769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a47bab-176d-4cd9-8f6e-9fe8e8e4d017 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:49.813042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 536/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:08:49 2022] PUT /identity/v3/projects/fe379f778baa42ec8af8518892165fad/users/ec0d0c3383814a1192b653b1fbdf1042/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:08:51.799726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924d76-3197-4e10-a08d-3439fdd4a844 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:51.800312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924d76-3197-4e10-a08d-3439fdd4a844 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:51.800312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c924d76-3197-4e10-a08d-3439fdd4a844 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:51.801953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 549/1085] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:51.808261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7729e-d63f-411f-8666-583c437490a7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:51.808814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7729e-d63f-411f-8666-583c437490a7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:51.808814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7729e-d63f-411f-8666-583c437490a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:51.826533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-25a7729e-d63f-411f-8666-583c437490a7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:51.827733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-25a7729e-d63f-411f-8666-583c437490a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:51.884214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:51.894080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 537/1086] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:51.918398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924d76-3197-4e10-a08d-3439fdd4a844 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:51.938139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:51.982355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5950-48ee-441e-a5f9-4a440f6a8f7f None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:51.982563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5950-48ee-441e-a5f9-4a440f6a8f7f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:51.982844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5950-48ee-441e-a5f9-4a440f6a8f7f None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:51.984394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 538/1087] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:08:51.989071 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc3c9c-1e30-42be-92df-fdd15ef9f004 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:51.989364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc3c9c-1e30-42be-92df-fdd15ef9f004 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:51.989644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc3c9c-1e30-42be-92df-fdd15ef9f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.006521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bedc3c9c-1e30-42be-92df-fdd15ef9f004 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:52.007717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bedc3c9c-1e30-42be-92df-fdd15ef9f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:52.021355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c924d76-3197-4e10-a08d-3439fdd4a844 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.054463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.055267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.056101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.056413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.067173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:52.076493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:52.077452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:52.083347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 550/1088] 10.222.0.42 () {56 vars in 1120 bytes} [Thu May 26 21:08:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:52.084112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 539/1089] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:52.084187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.103246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc3c9c-1e30-42be-92df-fdd15ef9f004 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.175424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:52.199005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7940c7b-b589-460d-b827-5a128f77977b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.210457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc3c9c-1e30-42be-92df-fdd15ef9f004 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.236777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.236978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:52.237076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.237293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.237652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.238345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:52.238681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.240107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.249311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.261210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.261210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.261210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.261514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 551/1090] 10.222.0.42 () {56 vars in 1115 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:52.261514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.262646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:08:52.270643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:52.271856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:52.272759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 540/1091] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:52.278883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.361807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ad632-98fa-4f28-8fe5-6422a7d5b523 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.395737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe113c9a-040e-42db-8fdc-d7b46d9b3539 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.397770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.398578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.398862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.399140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.408674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:52.410851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.412349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:52.429669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:52.431215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:52.432289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:52.432289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:52.432289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:52.432289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:52.432688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.432788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 552/1092] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:52.432858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.432858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.432858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.433434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:52.433539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.434851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.437176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.448731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 541/1093] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:52.449779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.496912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:52.542791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e7cbb-3d3a-4893-8104-826f27309948 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.557222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bbfabb-94d2-4550-ba26-1ef251b3e580 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.578572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.579233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.579514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.579824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.580548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:52.580852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.582221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.592012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.592941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.593225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.593498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] PATH_INFO: `/v3/users/ec0d0c3383814a1192b653b1fbdf1042` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.594576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 553/1094] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:52.594717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] RBAC: Authorizing `identity:get_user(user_id=ec0d0c3383814a1192b653b1fbdf1042)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.596248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.602672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 542/1095] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/users/ec0d0c3383814a1192b653b1fbdf1042 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:52.602852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.603729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.653004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:52.699317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84f06c-ab2b-44b7-8079-0664432071b0 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.702692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72120bf8-7c8a-49b6-972b-938104f7c434 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.715806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:52.734596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.735387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.735669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.735976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.736434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.737076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.737358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.737633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.743652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.745118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:08:52.760196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:52.761314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:08:52.762496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 554/1096] 10.222.0.42 () {56 vars in 1093 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:52.767426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.808685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.810189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.865273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08c178-675b-4dbb-b22a-351f0fafc527 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:52.878717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:52.899469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:52.900166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:52.900508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:52.900791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:52.901631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:52.901949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:52.903477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:52.908287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 543/1097] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 21:08:52.914000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.925668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 555/1098] 10.222.0.42 () {56 vars in 1133 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:52.927033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:52.927522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:53.012885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1357769-8c28-42d1-b84f-139450cba20d None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:53.025658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f24ab-4a3e-46b6-9f38-1f9e4e1ecbb9 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:53.047674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:53.049800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:53.049800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:53.049800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:53.050278 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] RBAC: Authorizing `identity:get_project(project_id=fe379f778baa42ec8af8518892165fad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:53.051723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:53.053431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 544/1099] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/projects/fe379f778baa42ec8af8518892165fad => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:08:53.059927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:53.061341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:53.061999 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:53.062289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:53.062569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:53.102436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:53.129165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:53.130628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:53.141441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:53.160425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d72a66-3ccc-4a6e-b7fb-bac0cd448b0f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:53.196013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:53.196698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:53.196988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:53.197260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:53.198086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=fe379f778baa42ec8af8518892165fad, user.id=ec0d0c3383814a1192b653b1fbdf1042) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:53.198399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:53.199841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:53.205670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 545/1100] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:53 2022] GET /identity/v3/role_assignments?user.id=ec0d0c3383814a1192b653b1fbdf1042&scope.project.id=fe379f778baa42ec8af8518892165fad&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:53.232965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 556/1101] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 21:08:53.238992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:53.345261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e3b986-74ae-4d5d-accf-337730338ea0 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:53.382881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:53.383714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:53.384025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:53.384302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:53.395450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:53.397024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:53.400633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:08:53.415806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:53.418380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:53.419317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:08:53.419317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:08:53.419317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:08:53.419317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:53.419317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 546/1102] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:08:53.422769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:53.493613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:53.522714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ace33-a966-4fc7-88ee-cbfd6215b3e8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:53.558684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:53.559346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:53.559628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:53.559936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:53.560653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:53.560959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:53.562320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:53.574318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 557/1103] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:53.579895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:53.677174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883e618b-d13a-4886-a4af-1d37592357e8 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:53.714241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:53.715033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:53.715314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:53.715584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:53.722239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:53.723662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:08:53.737268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:08:53.738156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:08:53.739138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 547/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:08:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 21:08:53.744968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:53.842946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773b612e-6dba-4016-a836-70e362d98a79 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:53.870851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:53.876415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e348df-07f5-4656-a3f4-2aec0b2720ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:53.877068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e348df-07f5-4656-a3f4-2aec0b2720ee None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:53.877350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e348df-07f5-4656-a3f4-2aec0b2720ee None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:53.877624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e348df-07f5-4656-a3f4-2aec0b2720ee None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:53.878507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e348df-07f5-4656-a3f4-2aec0b2720ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:53.878858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e348df-07f5-4656-a3f4-2aec0b2720ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:53.880323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e348df-07f5-4656-a3f4-2aec0b2720ee None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:53.891788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 558/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:08:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:53.897879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:53.954478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:53.997518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0a66fe-77b8-4455-9c80-56864008639c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:54.031172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010942f-792c-4651-8524-05a7f412b7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:54.031853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010942f-792c-4651-8524-05a7f412b7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:54.032136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010942f-792c-4651-8524-05a7f412b7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:54.032410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010942f-792c-4651-8524-05a7f412b7b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:54.033244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010942f-792c-4651-8524-05a7f412b7b8 None admin] RBAC: Adding query filter params (role.id=13a9265b6efa4d9da45dade8430e7d87, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=eefd00457d354271867932d336d7f69c) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:54.033554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010942f-792c-4651-8524-05a7f412b7b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:54.034978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010942f-792c-4651-8524-05a7f412b7b8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:54.040900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 548/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:53 2022] GET /identity/v3/role_assignments?user.id=eefd00457d354271867932d336d7f69c&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=13a9265b6efa4d9da45dade8430e7d87 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:55.447231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623a17a-4773-45e4-9544-e86505c2b620 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:55.447885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623a17a-4773-45e4-9544-e86505c2b620 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:55.447885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9623a17a-4773-45e4-9544-e86505c2b620 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:55.449298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 559/1107] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:55.454672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce4a27-d2fc-40ae-aab9-583857527d25 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:55.455088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce4a27-d2fc-40ae-aab9-583857527d25 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:55.455223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce4a27-d2fc-40ae-aab9-583857527d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:55.472466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-e9ce4a27-d2fc-40ae-aab9-583857527d25 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:55.473655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e9ce4a27-d2fc-40ae-aab9-583857527d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:55.539022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 549/1108] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:08:55.564302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623a17a-4773-45e4-9544-e86505c2b620 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:55.664486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9623a17a-4773-45e4-9544-e86505c2b620 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:55.698366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a3331-fd57-4a2b-ad62-f4f6c1a14b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:55.699089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3331-fd57-4a2b-ad62-f4f6c1a14b08 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:55.699365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3331-fd57-4a2b-ad62-f4f6c1a14b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:55.699638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a3331-fd57-4a2b-ad62-f4f6c1a14b08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:55.700835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a3331-fd57-4a2b-ad62-f4f6c1a14b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:55.702300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a3331-fd57-4a2b-ad62-f4f6c1a14b08 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:55.704227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 560/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:08:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:55.709754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce4a27-d2fc-40ae-aab9-583857527d25 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:55.758898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:55.806087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce4a27-d2fc-40ae-aab9-583857527d25 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:55.840489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0437bbe-c7e1-4e6b-9847-d5372956f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:55.841153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0437bbe-c7e1-4e6b-9847-d5372956f97c None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:55.841433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0437bbe-c7e1-4e6b-9847-d5372956f97c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:55.841706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0437bbe-c7e1-4e6b-9847-d5372956f97c None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:55.842416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0437bbe-c7e1-4e6b-9847-d5372956f97c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:55.843868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0437bbe-c7e1-4e6b-9847-d5372956f97c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:55.937458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 550/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:08:55 2022] POST /identity/v3/users => generated 286 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:08:58.144074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64c6a-1150-4a31-b2f2-f1aea58a88dd None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:58.144581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64c6a-1150-4a31-b2f2-f1aea58a88dd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:58.144581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e64c6a-1150-4a31-b2f2-f1aea58a88dd None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:58.145894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 561/1111] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:08:58.151226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d970d7-6ef0-4f1e-8fb1-5b5f274bce41 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:58.151603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d970d7-6ef0-4f1e-8fb1-5b5f274bce41 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:58.151821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d970d7-6ef0-4f1e-8fb1-5b5f274bce41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:58.168151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b7d970d7-6ef0-4f1e-8fb1-5b5f274bce41 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:08:58.169330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b7d970d7-6ef0-4f1e-8fb1-5b5f274bce41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:08:58.235696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 551/1112] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:08:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:08:58.258859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64c6a-1150-4a31-b2f2-f1aea58a88dd None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:58.316931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:58.358951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e64c6a-1150-4a31-b2f2-f1aea58a88dd None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:58.391700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:58.392573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:58.392858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:58.393138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] PATH_INFO: `/v3/roles/c23de713167345639a2085add0a54ae6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:58.394645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] RBAC: Authorizing `identity:get_role(role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:58.395992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:58.397413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 562/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:08:58 2022] GET /identity/v3/roles/c23de713167345639a2085add0a54ae6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:58.403766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d970d7-6ef0-4f1e-8fb1-5b5f274bce41 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:58.436857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:58.500310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d970d7-6ef0-4f1e-8fb1-5b5f274bce41 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:58.534137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:58.534980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:58.535261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:58.535567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] PATH_INFO: `/v3/users/66d63ec463e74fc18850ef4f9a5dac14` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:58.549173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] RBAC: Authorizing `identity:get_user(user_id=66d63ec463e74fc18850ef4f9a5dac14)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:58.551497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:58.553382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 552/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:08:58 2022] GET /identity/v3/users/66d63ec463e74fc18850ef4f9a5dac14 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:58.559836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:58.657015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a36635-03c3-4fd0-a446-9ba432e01a3e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:58.682700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:58.691059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:58.691698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:58.692017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:58.692296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:58.759760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:58.761250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:58.856099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 563/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:08:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:08:58.861952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:58.953863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:58.960553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf56a9-df5d-4850-8840-3e912bf9a2bf None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:58.993829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8c351-f917-46a0-89e9-4630250438bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:58.994672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c351-f917-46a0-89e9-4630250438bf None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:58.994951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c351-f917-46a0-89e9-4630250438bf None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:58.995222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8c351-f917-46a0-89e9-4630250438bf None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:58.996398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8c351-f917-46a0-89e9-4630250438bf None admin] RBAC: Authorizing `identity:get_project(project_id=fe379f778baa42ec8af8518892165fad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:58.997813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8c351-f917-46a0-89e9-4630250438bf None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:58.999470 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 553/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:08:58 2022] GET /identity/v3/projects/fe379f778baa42ec8af8518892165fad => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:08:59.005539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:08:59.032965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:08:59.105362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf35984-9f62-4bee-b93f-67b621951438 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:08:59.143126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e1d2d-0efd-478e-b70e-b7e943a47a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:08:59.143806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e1d2d-0efd-478e-b70e-b7e943a47a3d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:08:59.144094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e1d2d-0efd-478e-b70e-b7e943a47a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:08:59.144627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e1d2d-0efd-478e-b70e-b7e943a47a3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:08:59.145186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e1d2d-0efd-478e-b70e-b7e943a47a3d None admin] RBAC: Adding query filter params (role.id=c23de713167345639a2085add0a54ae6, scope.project.id=fe379f778baa42ec8af8518892165fad, user.id=66d63ec463e74fc18850ef4f9a5dac14) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:08:59.145501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e1d2d-0efd-478e-b70e-b7e943a47a3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:08:59.146940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e1d2d-0efd-478e-b70e-b7e943a47a3d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:08:59.153026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 564/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:08:59 2022] GET /identity/v3/role_assignments?user.id=66d63ec463e74fc18850ef4f9a5dac14&scope.project.id=fe379f778baa42ec8af8518892165fad&role.id=c23de713167345639a2085add0a54ae6 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:01.316393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6690b6-6480-48f9-ab2e-5a96c86938f2 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:01.316998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6690b6-6480-48f9-ab2e-5a96c86938f2 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:01.316998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6690b6-6480-48f9-ab2e-5a96c86938f2 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:01.318289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 554/1118] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:01.324246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae57530-02cf-487a-93cc-78ed4dd19f79 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:01.324685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae57530-02cf-487a-93cc-78ed4dd19f79 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:01.324785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae57530-02cf-487a-93cc-78ed4dd19f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:01.341722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-9ae57530-02cf-487a-93cc-78ed4dd19f79 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:01.342902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-9ae57530-02cf-487a-93cc-78ed4dd19f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:01.407093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 565/1119] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:01.429455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6690b6-6480-48f9-ab2e-5a96c86938f2 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:01.529576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6690b6-6480-48f9-ab2e-5a96c86938f2 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:01.562251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:01.563090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:01.563364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:01.563633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] PATH_INFO: `/v3/roles/c23de713167345639a2085add0a54ae6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:01.565148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] RBAC: Authorizing `identity:get_role(role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:01.566536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:01.567854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 555/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:01 2022] GET /identity/v3/roles/c23de713167345639a2085add0a54ae6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:01.574618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae57530-02cf-487a-93cc-78ed4dd19f79 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:01.595812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:01.671680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae57530-02cf-487a-93cc-78ed4dd19f79 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:01.705384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:01.706250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:01.706526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:01.706794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] PATH_INFO: `/v3/users/66d63ec463e74fc18850ef4f9a5dac14` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:01.708110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] RBAC: Authorizing `identity:get_user(user_id=66d63ec463e74fc18850ef4f9a5dac14)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:01.709553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:01.711255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 566/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:01 2022] GET /identity/v3/users/66d63ec463e74fc18850ef4f9a5dac14 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:01.717467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:01.747887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:01.815851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518de0c3-d29e-4ba9-a528-0e8a54756d80 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:01.849584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e3864-3e8d-4e69-8757-78a67bb543fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:01.850398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3864-3e8d-4e69-8757-78a67bb543fc None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:01.850676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3864-3e8d-4e69-8757-78a67bb543fc None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:01.850945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e3864-3e8d-4e69-8757-78a67bb543fc None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:01.852142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e3864-3e8d-4e69-8757-78a67bb543fc None admin] RBAC: Authorizing `identity:get_project(project_id=fe379f778baa42ec8af8518892165fad)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:01.853594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e3864-3e8d-4e69-8757-78a67bb543fc None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:01.855286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 556/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:01 2022] GET /identity/v3/projects/fe379f778baa42ec8af8518892165fad => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:01.861132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:01.945409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:01.957784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b0c52-b77e-4478-94ef-3237878c82b6 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:01.990583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53f123-7abe-4716-bf82-08eb0791d314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:01.991470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53f123-7abe-4716-bf82-08eb0791d314 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:01.991776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53f123-7abe-4716-bf82-08eb0791d314 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:01.992061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53f123-7abe-4716-bf82-08eb0791d314 None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad/users/66d63ec463e74fc18850ef4f9a5dac14/roles/c23de713167345639a2085add0a54ae6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:01.994774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53f123-7abe-4716-bf82-08eb0791d314 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe379f778baa42ec8af8518892165fad, user_id=66d63ec463e74fc18850ef4f9a5dac14, role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:01.996255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53f123-7abe-4716-bf82-08eb0791d314 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:02.068169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 567/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:09:01 2022] PUT /identity/v3/projects/fe379f778baa42ec8af8518892165fad/users/66d63ec463e74fc18850ef4f9a5dac14/roles/c23de713167345639a2085add0a54ae6 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:04.298527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b99ca-15e1-4ae9-87ad-c46b4ee98d77 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:04.299376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b99ca-15e1-4ae9-87ad-c46b4ee98d77 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:04.299547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b99ca-15e1-4ae9-87ad-c46b4ee98d77 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:04.301003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 557/1124] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:04.307276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b6ffb-0d19-4a79-b9db-e6dd8ff1544a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:04.307737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b6ffb-0d19-4a79-b9db-e6dd8ff1544a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:04.307857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b6ffb-0d19-4a79-b9db-e6dd8ff1544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:04.324569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-643b6ffb-0d19-4a79-b9db-e6dd8ff1544a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:04.325777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-643b6ffb-0d19-4a79-b9db-e6dd8ff1544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:04.393719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 568/1125] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:04.417922 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b99ca-15e1-4ae9-87ad-c46b4ee98d77 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:04.514409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:04.518958 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b99ca-15e1-4ae9-87ad-c46b4ee98d77 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:04.553476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:04.554242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:04.554523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:04.554796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] PATH_INFO: `/v3/roles/c23de713167345639a2085add0a54ae6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:04.556369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] RBAC: Authorizing `identity:get_role(role_id=c23de713167345639a2085add0a54ae6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:04.557733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:04.559153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 558/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:04 2022] GET /identity/v3/roles/c23de713167345639a2085add0a54ae6 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:04.565130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b6ffb-0d19-4a79-b9db-e6dd8ff1544a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:04.646187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:04.668412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b6ffb-0d19-4a79-b9db-e6dd8ff1544a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:04.705332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:04.706240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:04.706966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:04.706966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] PATH_INFO: `/v3/users/66d63ec463e74fc18850ef4f9a5dac14` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:04.708016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] RBAC: Authorizing `identity:get_user(user_id=66d63ec463e74fc18850ef4f9a5dac14)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:04.709475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:04.711368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 569/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:04 2022] GET /identity/v3/users/66d63ec463e74fc18850ef4f9a5dac14 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:04.717930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:04.820365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e42e3f-01b2-4652-b52e-3e6453c23651 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:04.855037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:04.855690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:04.856010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:04.856289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:04.893262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:04.927389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:04.928890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:05.034827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 559/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:09:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:09:05.041571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:05.144340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a6465-3dc7-4cfa-b743-ec2cd708917e None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:05.181754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc4d04-019f-42f3-b70f-2eefd00a838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:05.182708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc4d04-019f-42f3-b70f-2eefd00a838c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:05.183079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc4d04-019f-42f3-b70f-2eefd00a838c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:05.183397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc4d04-019f-42f3-b70f-2eefd00a838c None admin] PATH_INFO: `/v3/projects/fe379f778baa42ec8af8518892165fad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:05.184721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc4d04-019f-42f3-b70f-2eefd00a838c None admin] RBAC: Authorizing `identity:get_project(project_id=fe379f778baa42ec8af8518892165fad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:05.186222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc4d04-019f-42f3-b70f-2eefd00a838c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:05.188043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 570/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:05 2022] GET /identity/v3/projects/fe379f778baa42ec8af8518892165fad => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:05.194674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:05.288189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:05.346140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30d4353-80ae-4557-a53e-0dd7fc38051a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:05.402196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0189a-37ec-48c2-87b6-b2567ae8cb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:05.403193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0189a-37ec-48c2-87b6-b2567ae8cb42 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:05.403680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0189a-37ec-48c2-87b6-b2567ae8cb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:05.404189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0189a-37ec-48c2-87b6-b2567ae8cb42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:05.405819 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0189a-37ec-48c2-87b6-b2567ae8cb42 None admin] RBAC: Adding query filter params (role.id=c23de713167345639a2085add0a54ae6, scope.project.id=fe379f778baa42ec8af8518892165fad, user.id=66d63ec463e74fc18850ef4f9a5dac14) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:05.406368 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0189a-37ec-48c2-87b6-b2567ae8cb42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:05.409092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0189a-37ec-48c2-87b6-b2567ae8cb42 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:05.419339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 560/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:09:05 2022] GET /identity/v3/role_assignments?user.id=66d63ec463e74fc18850ef4f9a5dac14&scope.project.id=fe379f778baa42ec8af8518892165fad&role.id=c23de713167345639a2085add0a54ae6 => generated 608 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:07.679834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ab19-4fac-4c07-8900-e8cd4769f629 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:07.680422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ab19-4fac-4c07-8900-e8cd4769f629 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:07.680422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135ab19-4fac-4c07-8900-e8cd4769f629 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:07.681742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 571/1131] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:07.688017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25092def-f421-4bb8-9e48-376a7240f26f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:07.688719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25092def-f421-4bb8-9e48-376a7240f26f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:07.688931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25092def-f421-4bb8-9e48-376a7240f26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:07.707273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-25092def-f421-4bb8-9e48-376a7240f26f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:07.708603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-25092def-f421-4bb8-9e48-376a7240f26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:07.779548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 561/1132] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:07.803471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135ab19-4fac-4c07-8900-e8cd4769f629 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:07.820787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:07.909763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135ab19-4fac-4c07-8900-e8cd4769f629 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:07.945082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e63b4d0-8051-46e2-86fa-b405f80e398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:07.945891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63b4d0-8051-46e2-86fa-b405f80e398a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:07.946172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63b4d0-8051-46e2-86fa-b405f80e398a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:07.946447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e63b4d0-8051-46e2-86fa-b405f80e398a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:07.947719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63b4d0-8051-46e2-86fa-b405f80e398a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:07.949254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e63b4d0-8051-46e2-86fa-b405f80e398a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:07.950902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 572/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:09:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:07.957165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25092def-f421-4bb8-9e48-376a7240f26f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:08.017713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:08.062544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25092def-f421-4bb8-9e48-376a7240f26f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:08.099143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24409a1e-5a51-4df2-a232-745be93f79b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:08.099863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409a1e-5a51-4df2-a232-745be93f79b0 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:08.100123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409a1e-5a51-4df2-a232-745be93f79b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:08.100400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24409a1e-5a51-4df2-a232-745be93f79b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:08.101122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24409a1e-5a51-4df2-a232-745be93f79b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:08.102593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24409a1e-5a51-4df2-a232-745be93f79b0 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:08.203081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 562/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:09:07 2022] POST /identity/v3/projects => generated 311 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:09:10.482358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86dc0f-93f8-4943-b4c3-0627e4ddbff7 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:10.482992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86dc0f-93f8-4943-b4c3-0627e4ddbff7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:10.482992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86dc0f-93f8-4943-b4c3-0627e4ddbff7 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:10.484386 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 573/1135] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:10.489685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef25a-3f32-4380-ab04-c4186e7b921e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:10.490114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef25a-3f32-4380-ab04-c4186e7b921e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:10.490251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ef25a-3f32-4380-ab04-c4186e7b921e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:10.507179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-993ef25a-3f32-4380-ab04-c4186e7b921e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:10.508406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-993ef25a-3f32-4380-ab04-c4186e7b921e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:10.575064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 563/1136] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:10.600134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86dc0f-93f8-4943-b4c3-0627e4ddbff7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:10.682516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:10.701459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86dc0f-93f8-4943-b4c3-0627e4ddbff7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:10.735615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3e055-3e24-493b-848e-ccde7f292869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:10.736369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3e055-3e24-493b-848e-ccde7f292869 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:10.736656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3e055-3e24-493b-848e-ccde7f292869 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:10.736935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3e055-3e24-493b-848e-ccde7f292869 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:10.738120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3e055-3e24-493b-848e-ccde7f292869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:10.739694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3e055-3e24-493b-848e-ccde7f292869 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:10.741380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 574/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:09:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:10.747663 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef25a-3f32-4380-ab04-c4186e7b921e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:10.789170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:10.843601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ef25a-3f32-4380-ab04-c4186e7b921e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:10.877131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa048a-72d6-4340-aa9b-d54ce953f410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:10.877784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa048a-72d6-4340-aa9b-d54ce953f410 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:10.878102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa048a-72d6-4340-aa9b-d54ce953f410 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:10.878381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa048a-72d6-4340-aa9b-d54ce953f410 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:10.879092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa048a-72d6-4340-aa9b-d54ce953f410 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:10.880533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa048a-72d6-4340-aa9b-d54ce953f410 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:10.988322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 564/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:09:10 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:13.223095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcddb0f-cec9-4d76-b7fd-9971afd9400a None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:13.223804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcddb0f-cec9-4d76-b7fd-9971afd9400a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:13.223804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcddb0f-cec9-4d76-b7fd-9971afd9400a None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:13.225117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 575/1139] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:13.231034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9d9c-02f9-4076-b1f9-04aae64222fe None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:13.231441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9d9c-02f9-4076-b1f9-04aae64222fe None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:13.231595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf9d9c-02f9-4076-b1f9-04aae64222fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:13.248225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-dfaf9d9c-02f9-4076-b1f9-04aae64222fe None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:13.249399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dfaf9d9c-02f9-4076-b1f9-04aae64222fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:13.314806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 565/1140] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:13.339102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcddb0f-cec9-4d76-b7fd-9971afd9400a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:13.440416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcddb0f-cec9-4d76-b7fd-9971afd9400a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:13.473554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:13.474270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:13.474553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:13.474824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:13.485111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:09:13.493427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:09:13.494652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:09:13.495580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 576/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:09:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:09:13.500581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9d9c-02f9-4076-b1f9-04aae64222fe None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:13.527329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:13.599520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf9d9c-02f9-4076-b1f9-04aae64222fe None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:13.634149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:13.634806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:13.635127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:13.635363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:13.636141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:13.636447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:13.637801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:13.645915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 566/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:09:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:13.652443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:13.666089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:13.751918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05cb3c-cb2b-4217-9957-03f9c8bec447 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:13.784779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:13.785647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:13.785971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:13.786288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] PATH_INFO: `/v3/users/7a1b7e7665fa4bb8af9d30202e07218f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:13.798868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] RBAC: Authorizing `identity:get_user(user_id=7a1b7e7665fa4bb8af9d30202e07218f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:13.800391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:13.802019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 577/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:13 2022] GET /identity/v3/users/7a1b7e7665fa4bb8af9d30202e07218f => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:13.808434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:13.897539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:13.906512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5fcf4-a120-4d14-8d7b-3203f5151129 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:13.940108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:13.940741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:13.941031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:13.941310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:14.006994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:14.008422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:14.094399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:14.104426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 567/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:09:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:09:14.112073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:14.174593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:14.207193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac06a56-dd76-4b7c-a5a5-f2206cfd476d None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:14.241323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a21b3-b858-4c00-b2b7-42f73ecc924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:14.242173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a21b3-b858-4c00-b2b7-42f73ecc924b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:14.242457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a21b3-b858-4c00-b2b7-42f73ecc924b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:14.242728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a21b3-b858-4c00-b2b7-42f73ecc924b None admin] PATH_INFO: `/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:14.243936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a21b3-b858-4c00-b2b7-42f73ecc924b None admin] RBAC: Authorizing `identity:get_project(project_id=8d8e3d4f71b94ee0b1f6c15f8c129415)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:14.245370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a21b3-b858-4c00-b2b7-42f73ecc924b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:14.247120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 578/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:14 2022] GET /identity/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:14.253374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:14.350357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a88e7-cb57-4094-858d-85dbce2f42c9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:14.382287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874edbfe-2d17-4ef3-9ef1-c94faac0022c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:14.382938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874edbfe-2d17-4ef3-9ef1-c94faac0022c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:14.383772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874edbfe-2d17-4ef3-9ef1-c94faac0022c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:14.384053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874edbfe-2d17-4ef3-9ef1-c94faac0022c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:14.384876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874edbfe-2d17-4ef3-9ef1-c94faac0022c None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=8d8e3d4f71b94ee0b1f6c15f8c129415, user.id=7a1b7e7665fa4bb8af9d30202e07218f) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:14.385189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874edbfe-2d17-4ef3-9ef1-c94faac0022c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:14.386604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874edbfe-2d17-4ef3-9ef1-c94faac0022c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:14.392005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 568/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:09:14 2022] GET /identity/v3/role_assignments?user.id=7a1b7e7665fa4bb8af9d30202e07218f&scope.project.id=8d8e3d4f71b94ee0b1f6c15f8c129415&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:16.561240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f12d-fe3b-425d-bae8-5d4547f56b27 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:16.561887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f12d-fe3b-425d-bae8-5d4547f56b27 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:16.561887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6291f12d-fe3b-425d-bae8-5d4547f56b27 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:16.563165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 579/1147] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:16.568547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77144786-b0af-49a6-a035-de3d86fb015f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:16.568968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77144786-b0af-49a6-a035-de3d86fb015f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:16.569364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77144786-b0af-49a6-a035-de3d86fb015f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:16.587272 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-77144786-b0af-49a6-a035-de3d86fb015f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:16.588530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-77144786-b0af-49a6-a035-de3d86fb015f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:16.637369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:16.654557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 569/1148] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:16.677275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291f12d-fe3b-425d-bae8-5d4547f56b27 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:16.779155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6291f12d-fe3b-425d-bae8-5d4547f56b27 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:16.813789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:16.814507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:16.814790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:16.815066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:16.817746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:16.827085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:09:16.835157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:09:16.836958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:09:16.837959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 580/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:09:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:09:16.842679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77144786-b0af-49a6-a035-de3d86fb015f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:16.939719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77144786-b0af-49a6-a035-de3d86fb015f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:16.973698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:16.974366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:16.974648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:16.974925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:16.976623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:16.976623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:16.977911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:16.985842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 570/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:09:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:16.991795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:17.089245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7de2f3-3fd3-405f-ae8f-a725d0bd12d6 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:17.121233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:17.122093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:17.122373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:17.122644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] PATH_INFO: `/v3/users/7a1b7e7665fa4bb8af9d30202e07218f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:17.123839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] RBAC: Authorizing `identity:get_user(user_id=7a1b7e7665fa4bb8af9d30202e07218f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:17.125272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:17.126895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 581/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:16 2022] GET /identity/v3/users/7a1b7e7665fa4bb8af9d30202e07218f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:17.132675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:17.178211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:17.228883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c4473-fef2-4e7a-bb30-68896b5a210a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:17.261548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca807b92-22c9-44e5-b7e5-c4b6d68e9b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:17.262397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca807b92-22c9-44e5-b7e5-c4b6d68e9b39 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:17.262676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca807b92-22c9-44e5-b7e5-c4b6d68e9b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:17.262946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca807b92-22c9-44e5-b7e5-c4b6d68e9b39 None admin] PATH_INFO: `/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:17.264129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca807b92-22c9-44e5-b7e5-c4b6d68e9b39 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8e3d4f71b94ee0b1f6c15f8c129415)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:17.265556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca807b92-22c9-44e5-b7e5-c4b6d68e9b39 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:17.267220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 571/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:17 2022] GET /identity/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:17.273513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:17.334425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:17.372809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893d7eb-fe18-49c4-a424-6fc37cd6e731 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:17.406203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844467c8-ed8f-4699-8627-b0c0fd08f596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:17.407086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844467c8-ed8f-4699-8627-b0c0fd08f596 None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:17.407370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844467c8-ed8f-4699-8627-b0c0fd08f596 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:17.407644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844467c8-ed8f-4699-8627-b0c0fd08f596 None admin] PATH_INFO: `/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415/users/7a1b7e7665fa4bb8af9d30202e07218f/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:17.409685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844467c8-ed8f-4699-8627-b0c0fd08f596 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8e3d4f71b94ee0b1f6c15f8c129415, user_id=7a1b7e7665fa4bb8af9d30202e07218f, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:17.411121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844467c8-ed8f-4699-8627-b0c0fd08f596 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:17.526237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 582/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:09:17 2022] PUT /identity/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415/users/7a1b7e7665fa4bb8af9d30202e07218f/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:19.691612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d215049-efbb-4ee3-af83-e4fb45b08fd3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:19.692238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d215049-efbb-4ee3-af83-e4fb45b08fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:19.692238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d215049-efbb-4ee3-af83-e4fb45b08fd3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:19.693581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 572/1154] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:19.699252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eddf5b-acfa-4622-b5cc-6f3fb156e983 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:19.699692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eddf5b-acfa-4622-b5cc-6f3fb156e983 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:19.699845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eddf5b-acfa-4622-b5cc-6f3fb156e983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:19.717377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a5eddf5b-acfa-4622-b5cc-6f3fb156e983 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:19.724950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a5eddf5b-acfa-4622-b5cc-6f3fb156e983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:19.786273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 583/1155] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:19.810675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d215049-efbb-4ee3-af83-e4fb45b08fd3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:19.913789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d215049-efbb-4ee3-af83-e4fb45b08fd3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:19.926984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:19.947332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:19.948083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:19.948368 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:19.948642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:19.958897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:09:19.966979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:09:19.968147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:09:19.969350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 573/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:09:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:09:19.974285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eddf5b-acfa-4622-b5cc-6f3fb156e983 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:20.027600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:20.070503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eddf5b-acfa-4622-b5cc-6f3fb156e983 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:20.104871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:20.105527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:20.105809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:20.106079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:20.106815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:20.107121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:20.108515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:20.116281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 584/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:09:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:20.122668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:20.220080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e50b5a7-a7bc-4787-ae74-42da0f542bfb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:20.252448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:20.253331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:20.253610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:20.253880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] PATH_INFO: `/v3/users/7a1b7e7665fa4bb8af9d30202e07218f` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:20.255036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] RBAC: Authorizing `identity:get_user(user_id=7a1b7e7665fa4bb8af9d30202e07218f)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:20.256576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:20.258703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 574/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:20 2022] GET /identity/v3/users/7a1b7e7665fa4bb8af9d30202e07218f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:20.264463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:20.360835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e359cc8d-a488-47e4-a1ed-3a44fb920bbd None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:20.376037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:20.393546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:20.394184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:20.394463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:20.394732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:20.461559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:20.463010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:20.557306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 585/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:09:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:09:20.563045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:20.605482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:20.659163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e54b6-f0ce-40f8-9b97-b057ae2216bb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:20.691887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b93be75-95ae-40a5-99b2-8fd60d593bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:20.692727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93be75-95ae-40a5-99b2-8fd60d593bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:20.693005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93be75-95ae-40a5-99b2-8fd60d593bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:20.693280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b93be75-95ae-40a5-99b2-8fd60d593bb4 None admin] PATH_INFO: `/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:20.694702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93be75-95ae-40a5-99b2-8fd60d593bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8e3d4f71b94ee0b1f6c15f8c129415)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:20.696738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b93be75-95ae-40a5-99b2-8fd60d593bb4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:20.698412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 575/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:20 2022] GET /identity/v3/projects/8d8e3d4f71b94ee0b1f6c15f8c129415 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:20.704502 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:20.721957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:20.802928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a472d3d-186a-4208-b5e8-6160cff3cc7a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:20.837138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e56e3-0360-476f-85ee-f4db44af440b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:20.837722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e56e3-0360-476f-85ee-f4db44af440b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:20.838004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e56e3-0360-476f-85ee-f4db44af440b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:20.838282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e56e3-0360-476f-85ee-f4db44af440b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:20.839201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e56e3-0360-476f-85ee-f4db44af440b None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=8d8e3d4f71b94ee0b1f6c15f8c129415, user.id=7a1b7e7665fa4bb8af9d30202e07218f) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:20.839516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e56e3-0360-476f-85ee-f4db44af440b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:20.840995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e56e3-0360-476f-85ee-f4db44af440b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:20.846807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 586/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:09:20 2022] GET /identity/v3/role_assignments?user.id=7a1b7e7665fa4bb8af9d30202e07218f&scope.project.id=8d8e3d4f71b94ee0b1f6c15f8c129415&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:23.064015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480e7a7-1db7-4587-a64f-6dfc1a63a631 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:23.064538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480e7a7-1db7-4587-a64f-6dfc1a63a631 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:23.064538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e480e7a7-1db7-4587-a64f-6dfc1a63a631 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:23.065824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 576/1162] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:23.071870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a20b32-785c-4deb-a16c-e0b09efca944 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:23.072244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a20b32-785c-4deb-a16c-e0b09efca944 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:23.072432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a20b32-785c-4deb-a16c-e0b09efca944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:23.089779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d1a20b32-785c-4deb-a16c-e0b09efca944 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:23.090951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d1a20b32-785c-4deb-a16c-e0b09efca944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:23.155150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 587/1163] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:23.177308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480e7a7-1db7-4587-a64f-6dfc1a63a631 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:23.276676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e480e7a7-1db7-4587-a64f-6dfc1a63a631 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:23.287725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:23.312459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:23.313102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:23.313384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:23.313660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:23.386129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:23.387483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:23.483150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 577/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:09:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:09:23.489163 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a20b32-785c-4deb-a16c-e0b09efca944 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:23.501099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:23.588715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a20b32-785c-4deb-a16c-e0b09efca944 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:23.623602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76411c33-9ecc-47db-bdad-91e27a9b9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:23.624433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76411c33-9ecc-47db-bdad-91e27a9b9e35 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:23.624713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76411c33-9ecc-47db-bdad-91e27a9b9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:23.624990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76411c33-9ecc-47db-bdad-91e27a9b9e35 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:23.631441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76411c33-9ecc-47db-bdad-91e27a9b9e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:23.632953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76411c33-9ecc-47db-bdad-91e27a9b9e35 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-76411c33-9ecc-47db-bdad-91e27a9b9e35 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:09:23.646855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:09:23.647915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:09:23.649446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 588/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:09:23 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:09:23.654471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:23.666635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:23.754696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5ed8b-35b8-42f2-8540-c241b3985407 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:23.789735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e428c0-2c3e-4d6f-bf6b-169d992ff313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:23.790405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e428c0-2c3e-4d6f-bf6b-169d992ff313 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:23.790687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e428c0-2c3e-4d6f-bf6b-169d992ff313 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:23.790961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e428c0-2c3e-4d6f-bf6b-169d992ff313 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:23.791710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e428c0-2c3e-4d6f-bf6b-169d992ff313 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:23.792048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e428c0-2c3e-4d6f-bf6b-169d992ff313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:23.793423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e428c0-2c3e-4d6f-bf6b-169d992ff313 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:23.800646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 578/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Thu May 26 21:09:23 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:25.979412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e106a29-b4fa-4fca-9488-8f1def5ecfcf None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:25.979964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e106a29-b4fa-4fca-9488-8f1def5ecfcf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:25.979964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e106a29-b4fa-4fca-9488-8f1def5ecfcf None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:25.981324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 589/1167] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:25.986657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72492a-300b-412f-8715-4e278276b7eb None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:25.987037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72492a-300b-412f-8715-4e278276b7eb None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:25.987208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d72492a-300b-412f-8715-4e278276b7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:26.003978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2d72492a-300b-412f-8715-4e278276b7eb None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:26.005147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2d72492a-300b-412f-8715-4e278276b7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:26.071268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 579/1168] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:26.093988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e106a29-b4fa-4fca-9488-8f1def5ecfcf None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:26.162576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:26.193413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e106a29-b4fa-4fca-9488-8f1def5ecfcf None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:26.226212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a41ec7-c236-4a4b-9399-86eab852795f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:26.227389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41ec7-c236-4a4b-9399-86eab852795f None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:26.227670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41ec7-c236-4a4b-9399-86eab852795f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:26.227970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a41ec7-c236-4a4b-9399-86eab852795f None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:26.228704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a41ec7-c236-4a4b-9399-86eab852795f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:26.230116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a41ec7-c236-4a4b-9399-86eab852795f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:26.297842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 590/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Thu May 26 21:09:26 2022] POST /identity/v3/domains => generated 267 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:28.484464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3638003-0e4a-477e-afe8-ff7120a3b990 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:28.485067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3638003-0e4a-477e-afe8-ff7120a3b990 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:28.485067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3638003-0e4a-477e-afe8-ff7120a3b990 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:28.486350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 580/1170] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:28.491498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d13215-dac9-4de7-846e-7d6e3e7b72d1 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:28.491914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d13215-dac9-4de7-846e-7d6e3e7b72d1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:28.492070 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d13215-dac9-4de7-846e-7d6e3e7b72d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:28.509122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c5d13215-dac9-4de7-846e-7d6e3e7b72d1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:28.510405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c5d13215-dac9-4de7-846e-7d6e3e7b72d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:28.574287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 591/1171] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:28.597969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3638003-0e4a-477e-afe8-ff7120a3b990 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:28.601766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:28.697072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3638003-0e4a-477e-afe8-ff7120a3b990 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:28.730931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41355e27-4122-44d9-9bf7-ead3d13b8b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:28.731794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41355e27-4122-44d9-9bf7-ead3d13b8b80 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:28.732078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41355e27-4122-44d9-9bf7-ead3d13b8b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:28.732352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41355e27-4122-44d9-9bf7-ead3d13b8b80 None admin] PATH_INFO: `/v3/domains/0c91dfc33f0f41cf82bba9fe155ee616` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:28.743047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41355e27-4122-44d9-9bf7-ead3d13b8b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c91dfc33f0f41cf82bba9fe155ee616)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:28.744445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41355e27-4122-44d9-9bf7-ead3d13b8b80 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:28.746055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 581/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Thu May 26 21:09:28 2022] GET /identity/v3/domains/0c91dfc33f0f41cf82bba9fe155ee616 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:28.751891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d13215-dac9-4de7-846e-7d6e3e7b72d1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:28.810174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:28.849544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d13215-dac9-4de7-846e-7d6e3e7b72d1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:28.883079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99590a-1849-46b5-8da5-4448fc272b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:28.883760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99590a-1849-46b5-8da5-4448fc272b53 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:28.884048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99590a-1849-46b5-8da5-4448fc272b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:28.884325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99590a-1849-46b5-8da5-4448fc272b53 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:28.885028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99590a-1849-46b5-8da5-4448fc272b53 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:28.886482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99590a-1849-46b5-8da5-4448fc272b53 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:28.988731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 592/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:09:28 2022] POST /identity/v3/projects => generated 361 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:31.193135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322835d-5441-4902-944b-660831a4693c None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:31.193671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322835d-5441-4902-944b-660831a4693c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:31.193671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322835d-5441-4902-944b-660831a4693c None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:31.195293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 582/1174] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:31.200510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2b5de-36fd-425a-97bf-e3357953f0d3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:31.200966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2b5de-36fd-425a-97bf-e3357953f0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:31.201072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2b5de-36fd-425a-97bf-e3357953f0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:31.218211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0ae2b5de-36fd-425a-97bf-e3357953f0d3 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:31.219397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0ae2b5de-36fd-425a-97bf-e3357953f0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:31.286019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 593/1175] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:31.311469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322835d-5441-4902-944b-660831a4693c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:31.377015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:31.415808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322835d-5441-4902-944b-660831a4693c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:31.448869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ecdec-f43a-4bdc-a374-317d3fb62114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:31.449708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ecdec-f43a-4bdc-a374-317d3fb62114 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:31.449997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ecdec-f43a-4bdc-a374-317d3fb62114 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:31.450279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ecdec-f43a-4bdc-a374-317d3fb62114 None admin] PATH_INFO: `/v3/domains/0c91dfc33f0f41cf82bba9fe155ee616` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:31.451449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ecdec-f43a-4bdc-a374-317d3fb62114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c91dfc33f0f41cf82bba9fe155ee616)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:31.452930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ecdec-f43a-4bdc-a374-317d3fb62114 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:31.454542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 583/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Thu May 26 21:09:31 2022] GET /identity/v3/domains/0c91dfc33f0f41cf82bba9fe155ee616 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:31.460438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2b5de-36fd-425a-97bf-e3357953f0d3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:31.502744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:31.558162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2b5de-36fd-425a-97bf-e3357953f0d3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:31.593309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8572f7b-a7ca-444a-96bb-4594dcda087d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:31.593977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8572f7b-a7ca-444a-96bb-4594dcda087d None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:31.594263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8572f7b-a7ca-444a-96bb-4594dcda087d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:31.594542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8572f7b-a7ca-444a-96bb-4594dcda087d None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:31.595253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8572f7b-a7ca-444a-96bb-4594dcda087d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:31.596731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8572f7b-a7ca-444a-96bb-4594dcda087d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:31.686284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 594/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:09:31 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:09:33.907851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ab884-e573-45c6-af56-83b548f0a119 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:33.908379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ab884-e573-45c6-af56-83b548f0a119 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:33.908379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ab884-e573-45c6-af56-83b548f0a119 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:33.909756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 584/1178] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:33.915058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63674ed-e117-4142-8f6a-5a0bbbf8354c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:33.915461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63674ed-e117-4142-8f6a-5a0bbbf8354c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:33.915617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63674ed-e117-4142-8f6a-5a0bbbf8354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:33.933228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d63674ed-e117-4142-8f6a-5a0bbbf8354c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:33.934422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d63674ed-e117-4142-8f6a-5a0bbbf8354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:33.999562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 595/1179] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:34.028039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ab884-e573-45c6-af56-83b548f0a119 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:34.128545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ab884-e573-45c6-af56-83b548f0a119 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:34.157384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:34.162089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:34.162877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:34.163160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:34.163436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:34.173658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:09:34.181326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:09:34.182437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:09:34.183391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 585/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:09:34.188582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63674ed-e117-4142-8f6a-5a0bbbf8354c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:34.216176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:34.286751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63674ed-e117-4142-8f6a-5a0bbbf8354c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:34.321737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:34.322398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:34.322678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:34.322951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:34.323768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:34.324076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:34.325460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:34.334014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 596/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:34.340624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:34.440380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f3615-3669-4bb5-b03b-57d7fb49f100 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:34.473259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:34.474121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:34.474404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:34.474785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] PATH_INFO: `/v3/users/149f8e463c354eb69f7de04b72c4509a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:34.488982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] RBAC: Authorizing `identity:get_user(user_id=149f8e463c354eb69f7de04b72c4509a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:34.490438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:34.492674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 586/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:34 2022] GET /identity/v3/users/149f8e463c354eb69f7de04b72c4509a => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:34.498552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:34.587331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:34.596498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6254fce1-5190-4a15-bbfa-5c5096e37a30 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:34.629147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:34.629835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:34.630114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:34.630387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:34.697508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:34.698997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:34.787566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:34.797411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 597/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:09:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:09:34.803423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:34.853918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:34.902047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943fb84-7f30-4aa7-8293-270d2fca38ca None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:34.935702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faff15b-7223-4f11-9e90-aaacdcb754be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:34.936568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faff15b-7223-4f11-9e90-aaacdcb754be None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:34.936847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faff15b-7223-4f11-9e90-aaacdcb754be None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:34.937114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faff15b-7223-4f11-9e90-aaacdcb754be None admin] PATH_INFO: `/v3/projects/2dd16c7033514ea0bec4f4e659349f87` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:34.938267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faff15b-7223-4f11-9e90-aaacdcb754be None admin] RBAC: Authorizing `identity:get_project(project_id=2dd16c7033514ea0bec4f4e659349f87)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:34.939675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faff15b-7223-4f11-9e90-aaacdcb754be None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:34.941366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 587/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:34 2022] GET /identity/v3/projects/2dd16c7033514ea0bec4f4e659349f87 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:34.948519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:35.044919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cf3ab-4c2c-47aa-86a0-28771b9cd1ea None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:35.082449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8190fb8-70a4-45ce-9d95-0122f9e5e175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:35.083094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8190fb8-70a4-45ce-9d95-0122f9e5e175 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:35.083375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8190fb8-70a4-45ce-9d95-0122f9e5e175 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:35.083645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8190fb8-70a4-45ce-9d95-0122f9e5e175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:35.084497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8190fb8-70a4-45ce-9d95-0122f9e5e175 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2dd16c7033514ea0bec4f4e659349f87, user.id=149f8e463c354eb69f7de04b72c4509a) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:35.084803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8190fb8-70a4-45ce-9d95-0122f9e5e175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:35.086850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8190fb8-70a4-45ce-9d95-0122f9e5e175 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:35.092399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 598/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:09:34 2022] GET /identity/v3/role_assignments?user.id=149f8e463c354eb69f7de04b72c4509a&scope.project.id=2dd16c7033514ea0bec4f4e659349f87&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:37.267641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333f79-fef3-43f6-b78a-5589b3577fa8 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:37.268234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333f79-fef3-43f6-b78a-5589b3577fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:37.268234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee333f79-fef3-43f6-b78a-5589b3577fa8 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:37.269550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 588/1186] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:37.275205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40914627-3268-41bc-8da8-d95b6c11ed41 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:37.275599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40914627-3268-41bc-8da8-d95b6c11ed41 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:37.275764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40914627-3268-41bc-8da8-d95b6c11ed41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:37.292755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-40914627-3268-41bc-8da8-d95b6c11ed41 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:37.293925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-40914627-3268-41bc-8da8-d95b6c11ed41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:37.343115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:37.366236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 599/1187] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:37.388691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333f79-fef3-43f6-b78a-5589b3577fa8 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:37.489672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee333f79-fef3-43f6-b78a-5589b3577fa8 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:37.508256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:37.527039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:37.527800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:37.528092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:37.528372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:37.538200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:09:37.546299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:09:37.547439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:09:37.548372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 589/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:09:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:09:37.553611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40914627-3268-41bc-8da8-d95b6c11ed41 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:37.651632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40914627-3268-41bc-8da8-d95b6c11ed41 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:37.684590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:37.685246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:37.685526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:37.685796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:37.686530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:37.686833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:37.688234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:37.696921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 600/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:09:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:37.703358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:37.801284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe949fe7-5eda-4070-98e3-f589f17fa851 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:37.835054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:37.835968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:37.836254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:37.836529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] PATH_INFO: `/v3/users/149f8e463c354eb69f7de04b72c4509a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:37.837711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] RBAC: Authorizing `identity:get_user(user_id=149f8e463c354eb69f7de04b72c4509a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:37.839137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:37.840775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 590/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:37 2022] GET /identity/v3/users/149f8e463c354eb69f7de04b72c4509a => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:37.846738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:37.893012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:37.944930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e42c3c3-f1b4-4a12-acb5-dee63a234357 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:37.977337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f258b-33a2-417d-ac2d-cb02fd934cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:37.978194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f258b-33a2-417d-ac2d-cb02fd934cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:37.978480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f258b-33a2-417d-ac2d-cb02fd934cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:37.978754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f258b-33a2-417d-ac2d-cb02fd934cc3 None admin] PATH_INFO: `/v3/projects/2dd16c7033514ea0bec4f4e659349f87` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:37.979954 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f258b-33a2-417d-ac2d-cb02fd934cc3 None admin] RBAC: Authorizing `identity:get_project(project_id=2dd16c7033514ea0bec4f4e659349f87)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:37.981409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f258b-33a2-417d-ac2d-cb02fd934cc3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:37.983130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 601/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:37 2022] GET /identity/v3/projects/2dd16c7033514ea0bec4f4e659349f87 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:37.989219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:38.036023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:38.089362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb54c52-e7ea-41db-9d0e-46fdae207a7f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:38.121164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c8dc5e-fdea-4c2f-9e5c-3573da54a7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:38.122041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8dc5e-fdea-4c2f-9e5c-3573da54a7ba None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:38.122323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8dc5e-fdea-4c2f-9e5c-3573da54a7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:38.122599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c8dc5e-fdea-4c2f-9e5c-3573da54a7ba None admin] PATH_INFO: `/v3/projects/2dd16c7033514ea0bec4f4e659349f87/users/149f8e463c354eb69f7de04b72c4509a/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:38.124817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8dc5e-fdea-4c2f-9e5c-3573da54a7ba None admin] RBAC: Authorizing `identity:create_grant(project_id=2dd16c7033514ea0bec4f4e659349f87, user_id=149f8e463c354eb69f7de04b72c4509a, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:38.126353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c8dc5e-fdea-4c2f-9e5c-3573da54a7ba None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:38.239263 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 591/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:09:37 2022] PUT /identity/v3/projects/2dd16c7033514ea0bec4f4e659349f87/users/149f8e463c354eb69f7de04b72c4509a/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:09:40.429478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed995-9baa-4a72-bb63-c2fa648f38ae None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:40.430049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed995-9baa-4a72-bb63-c2fa648f38ae None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:40.430049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ed995-9baa-4a72-bb63-c2fa648f38ae None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:40.431384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 602/1193] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:09:40.436710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea018399-2f3a-497b-8a08-5e3356e65756 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:40.437152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea018399-2f3a-497b-8a08-5e3356e65756 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:40.437266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea018399-2f3a-497b-8a08-5e3356e65756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:40.454342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ea018399-2f3a-497b-8a08-5e3356e65756 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:09:40.455546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ea018399-2f3a-497b-8a08-5e3356e65756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:09:40.519859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 592/1194] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:09:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:09:40.543830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ed995-9baa-4a72-bb63-c2fa648f38ae None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:40.646011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ed995-9baa-4a72-bb63-c2fa648f38ae None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:40.660082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:40.679302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:40.680729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:40.680963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:40.681236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:40.691772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:09:40.700566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:09:40.701778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:09:40.702705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 603/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:09:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:09:40.708390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea018399-2f3a-497b-8a08-5e3356e65756 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:40.742448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:40.805909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea018399-2f3a-497b-8a08-5e3356e65756 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:40.840886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:40.841540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:40.841825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:40.842093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:40.842828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:40.843131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:40.844522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:40.852981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 593/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:40.858798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:40.958501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb95ea0e-d73f-4b46-8a04-e0865f0d3caf None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:40.991679 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:40.993107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:40.993391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:40.993663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] PATH_INFO: `/v3/users/149f8e463c354eb69f7de04b72c4509a` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:40.994840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] RBAC: Authorizing `identity:get_user(user_id=149f8e463c354eb69f7de04b72c4509a)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:40.996343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:40.997973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 604/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Thu May 26 21:09:40 2022] GET /identity/v3/users/149f8e463c354eb69f7de04b72c4509a => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:41.003832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:41.097361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:41.101034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5ee1b2-1cf6-4a85-a8ed-80062a074a57 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:41.134979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:41.135640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:41.136157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:41.136609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:41.202984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:41.204454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:41.296205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:41.300484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 594/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:09:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:09:41.307376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:41.350846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:41.408009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4faf6-6754-4f93-9421-4df823f4a06f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:41.442907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f63c393-9649-45ff-a073-d46b196c4b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:41.444043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63c393-9649-45ff-a073-d46b196c4b32 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:41.444329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63c393-9649-45ff-a073-d46b196c4b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:41.444604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63c393-9649-45ff-a073-d46b196c4b32 None admin] PATH_INFO: `/v3/projects/2dd16c7033514ea0bec4f4e659349f87` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:41.445781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63c393-9649-45ff-a073-d46b196c4b32 None admin] RBAC: Authorizing `identity:get_project(project_id=2dd16c7033514ea0bec4f4e659349f87)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:41.447210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f63c393-9649-45ff-a073-d46b196c4b32 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:41.448930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 605/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Thu May 26 21:09:41 2022] GET /identity/v3/projects/2dd16c7033514ea0bec4f4e659349f87 => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:41.455063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:09:41.554251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b7515-7fbc-4949-8830-7beacf32ffed None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:09:41.588074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de471bdb-f8f2-477a-81c9-58f2dc4cb7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:09:41.588731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de471bdb-f8f2-477a-81c9-58f2dc4cb7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:09:41.589010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de471bdb-f8f2-477a-81c9-58f2dc4cb7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:09:41.589286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de471bdb-f8f2-477a-81c9-58f2dc4cb7c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:09:41.590113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de471bdb-f8f2-477a-81c9-58f2dc4cb7c8 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2dd16c7033514ea0bec4f4e659349f87, user.id=149f8e463c354eb69f7de04b72c4509a) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:09:41.590424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de471bdb-f8f2-477a-81c9-58f2dc4cb7c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:09:41.591883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de471bdb-f8f2-477a-81c9-58f2dc4cb7c8 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:09:41.598046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 595/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:09:41 2022] GET /identity/v3/role_assignments?user.id=149f8e463c354eb69f7de04b72c4509a&scope.project.id=2dd16c7033514ea0bec4f4e659349f87&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:09:51.438792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=45 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:09:51.595347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=69 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:03.783985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fc754-8908-4cb6-aa11-f15378cd77d7 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:03.784565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fc754-8908-4cb6-aa11-f15378cd77d7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:03.784565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33fc754-8908-4cb6-aa11-f15378cd77d7 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:03.785964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 606/1201] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:03.792149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aec8e-cb91-40ef-91d0-a37223657de5 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:03.792547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aec8e-cb91-40ef-91d0-a37223657de5 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:03.792691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0aec8e-cb91-40ef-91d0-a37223657de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:03.809355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1e0aec8e-cb91-40ef-91d0-a37223657de5 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:03.810534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1e0aec8e-cb91-40ef-91d0-a37223657de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:03.877235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 596/1202] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:11:03.902964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fc754-8908-4cb6-aa11-f15378cd77d7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.005005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33fc754-8908-4cb6-aa11-f15378cd77d7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.038598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.040673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.040673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.040673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.050508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.051996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:04.069685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:04.070817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:04.072159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:04.072159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:04.072159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:04.072159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:04.072159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 607/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:04.076938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aec8e-cb91-40ef-91d0-a37223657de5 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.176282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0aec8e-cb91-40ef-91d0-a37223657de5 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.205512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:04.211285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ab260-4f50-44d7-99d7-d9c717420337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.211986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ab260-4f50-44d7-99d7-d9c717420337 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.212274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ab260-4f50-44d7-99d7-d9c717420337 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.212550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ab260-4f50-44d7-99d7-d9c717420337 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.213341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ab260-4f50-44d7-99d7-d9c717420337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:04.213620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ab260-4f50-44d7-99d7-d9c717420337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.215039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ab260-4f50-44d7-99d7-d9c717420337 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:04.226379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 597/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:04.232587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:04.287722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:04.332806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a626613-7fe4-4903-8975-2dcf492c22a3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:04.366605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f08320-cedd-447a-b9b6-50b7f6a7d2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:04.367270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f08320-cedd-447a-b9b6-50b7f6a7d2bb None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:04.367551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f08320-cedd-447a-b9b6-50b7f6a7d2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:04.367903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f08320-cedd-447a-b9b6-50b7f6a7d2bb None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:04.368606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f08320-cedd-447a-b9b6-50b7f6a7d2bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:04.370339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f08320-cedd-447a-b9b6-50b7f6a7d2bb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:04.501245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 608/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Thu May 26 21:11:04 2022] POST /identity/v3/users => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:06.745338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6371-6d2e-43df-aae8-714e5376e0a3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:06.745814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6371-6d2e-43df-aae8-714e5376e0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:06.745930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b6371-6d2e-43df-aae8-714e5376e0a3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:06.747249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 598/1206] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:06.752553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6922-6c1b-48e1-a7f2-65d0e9330052 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:06.752947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6922-6c1b-48e1-a7f2-65d0e9330052 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:06.753105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b6922-6c1b-48e1-a7f2-65d0e9330052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:06.770291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-dc5b6922-6c1b-48e1-a7f2-65d0e9330052 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:06.771481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-dc5b6922-6c1b-48e1-a7f2-65d0e9330052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:06.838555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 609/1207] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:06.861963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b6371-6d2e-43df-aae8-714e5376e0a3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:06.968009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b6371-6d2e-43df-aae8-714e5376e0a3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:07.000360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:07.001144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:07.001426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:07.001699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:07.012117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:11:07.019885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:07.021012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:11:07.022073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:07.023917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 599/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:11:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:07.027664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6922-6c1b-48e1-a7f2-65d0e9330052 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:07.066204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:07.123711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b6922-6c1b-48e1-a7f2-65d0e9330052 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:07.157283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:07.157941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:07.158225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:07.158500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:07.159249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] RBAC: Adding query filter params (name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:07.159565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:07.160982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:07.169242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 610/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:11:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:07.174999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:07.226184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:07.275775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87142b6-430a-4d85-bded-21e12a3b48a9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:07.308404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:07.309191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:07.309466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:07.309731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:07.320748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:07.322820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:07.339490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:07.340606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:07.341535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:07.341535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:07.341535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:07.341535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:07.341535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 600/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:07.346455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:07.442264 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:07.445415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b006e-9128-463c-b411-2ebcf06ac179 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:07.478384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:07.479524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:07.479835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:07.480125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:07.480832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:07.481140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:07.482543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:07.493891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 611/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:07.499600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:07.602206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939af2b-b3ed-427e-8dee-1bcc8fcac13f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:07.604427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:07.636780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:07.637577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:07.637861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:07.638137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:07.645138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:07.646577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:11:07.661904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:07.662924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:07.664051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 601/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:07.668896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:07.769221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325aff91-7ad7-414a-bf49-ebe20b75b5c2 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:07.802429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:07.803085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:07.803367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:07.803639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:07.804500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:07.804917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:07.806349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:07.829632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 612/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Thu May 26 21:11:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:07.835367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:07.933425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d2fbe8-af5d-4813-ab52-77e8e26a88d6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:07.967961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:07.968602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:07.968885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:07.969159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:07.998095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:08.036092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:08.037503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:08.133590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 602/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:11:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:11:08.139449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:08.158897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:08.237943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d8f01e-4e05-43e5-8ba8-d04ab1a12606 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:08.272171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:08.272959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:08.273238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:08.273510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:08.283683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:08.285176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:08.302993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:08.304068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:08.305252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:08.305252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:08.305252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:08.305252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:08.305252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 613/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:08.310860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:08.373458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:08.411350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cd2d8-eb07-4f36-a568-48932e5ab8fe None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:08.445268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:08.445924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:08.446203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:08.446474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:08.447698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:08.448064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:08.449403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:08.460687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 603/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:08.467104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:08.535123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:08.566059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c7f46-4219-4a9a-b1b8-33483b888de5 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:08.599662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:08.600492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:08.600781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:08.601061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:08.608453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:08.609918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:11:08.625110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:08.626218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:08.627309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 614/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:08.632408 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:08.737281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e727edc-b007-4cfb-a371-d01c71109747 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:08.760290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:08.771830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ccc277-047b-4a09-86d2-2cd8b9ee8880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:08.772504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc277-047b-4a09-86d2-2cd8b9ee8880 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:08.772786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc277-047b-4a09-86d2-2cd8b9ee8880 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:08.773067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ccc277-047b-4a09-86d2-2cd8b9ee8880 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:08.773898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccc277-047b-4a09-86d2-2cd8b9ee8880 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:08.774211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccc277-047b-4a09-86d2-2cd8b9ee8880 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:08.776184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ccc277-047b-4a09-86d2-2cd8b9ee8880 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:08.787400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 604/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:11:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:08.794102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:08.892833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316852f-e428-434c-9d4a-75049ac88ad7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:08.920830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:08.926513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7f121-c9e9-4a67-9add-8fd80e356103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:08.927160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7f121-c9e9-4a67-9add-8fd80e356103 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:08.927479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7f121-c9e9-4a67-9add-8fd80e356103 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:08.927769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7f121-c9e9-4a67-9add-8fd80e356103 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:08.928599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7f121-c9e9-4a67-9add-8fd80e356103 None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=fd73860db589439da5178e7010d567d2) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:08.928911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7f121-c9e9-4a67-9add-8fd80e356103 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:08.930337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7f121-c9e9-4a67-9add-8fd80e356103 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:08.935766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 615/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:11:08 2022] GET /identity/v3/role_assignments?user.id=fd73860db589439da5178e7010d567d2&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:11.093198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75355546-a00f-4553-9c40-1ae757e29fc6 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.093746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75355546-a00f-4553-9c40-1ae757e29fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.093746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75355546-a00f-4553-9c40-1ae757e29fc6 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.095214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 605/1220] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:11.100418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4cba-321a-4564-ac89-340b95f7faf9 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.100803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4cba-321a-4564-ac89-340b95f7faf9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.100967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b4cba-321a-4564-ac89-340b95f7faf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.117407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-872b4cba-321a-4564-ac89-340b95f7faf9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:11.118684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-872b4cba-321a-4564-ac89-340b95f7faf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:11.185997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 616/1221] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:11.209657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75355546-a00f-4553-9c40-1ae757e29fc6 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.311766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75355546-a00f-4553-9c40-1ae757e29fc6 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.347113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.347929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.348215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.348489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.358634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:11:11.366277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:11.367371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:11.369827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 606/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:11.374131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b4cba-321a-4564-ac89-340b95f7faf9 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.473495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b4cba-321a-4564-ac89-340b95f7faf9 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.497603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:11.508624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.509324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.509646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.509919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.510670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:11.510975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.512427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.520322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 617/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:11.526325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.571917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:11.634029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04a837-4d3e-48fe-b859-fbb401e4a4ee None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.667143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.667973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.668257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.668530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.678977 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.680442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:11.698204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:11.699451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:11.704772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:11.704772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:11.704772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:11.704772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:11.704772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 607/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:11.706478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.806977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f876148-40af-4c41-80dd-7d371653fbcc None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.841142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.841798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.842127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.842400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:11.843124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:11.843437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:11.844839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:11.857171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 618/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:11.862826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:11.956561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:11.963700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3d57b3-ff85-4cb0-85c4-ad27775f036f None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:11.997416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:11.998220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:11.998506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:11.998781 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.005625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.007054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:11:12.022743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:12.023910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:12.025152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 608/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:12.031121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.078566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:12.129608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd1836d-0374-42d0-9959-e93dfbcb32e6 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.164256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.164985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.165268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.165570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.166900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:12.167208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.168745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.185992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 619/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:11:12.191691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.289372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166b43a-9843-4bd3-9c55-2d3c94c1df97 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.323422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.324237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.324517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.324787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.334616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.336061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:12.353018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:12.354652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:12.355837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:12.355837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:12.355837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:12.355837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:12.355837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 609/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:12.360393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.459767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e03772-6ced-4802-9054-4be5cd70d4c5 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.462180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:12.493610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.497963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.497963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.497963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.497963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:12.497963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.497963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.509065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 620/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:12.514876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.523594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:12.614107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af87192d-03f4-461a-8c66-04f99f900ab6 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.647355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.648173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.648458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.648733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.655723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.657181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:11:12.671148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:12.672267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:12.673402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 610/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:12.678526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.780035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55989fd9-dbb8-4a93-9e44-2846148a858b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.813056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d419fb-54bd-40a3-b92c-4d14431b0beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.813730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d419fb-54bd-40a3-b92c-4d14431b0beb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.814010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d419fb-54bd-40a3-b92c-4d14431b0beb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.814281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d419fb-54bd-40a3-b92c-4d14431b0beb None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.815058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d419fb-54bd-40a3-b92c-4d14431b0beb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:12.815364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d419fb-54bd-40a3-b92c-4d14431b0beb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.816891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d419fb-54bd-40a3-b92c-4d14431b0beb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:12.829848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 621/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:11:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:12.836051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:12.903424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:12.935866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57bbe7-70ca-478e-9b82-7030fbbcbce4 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:12.969578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe7d10-6e00-4ef8-820e-4cd4e51009ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:12.970460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe7d10-6e00-4ef8-820e-4cd4e51009ce None admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:12.970743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe7d10-6e00-4ef8-820e-4cd4e51009ce None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:12.971020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe7d10-6e00-4ef8-820e-4cd4e51009ce None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/fd73860db589439da5178e7010d567d2/roles/7c5539dafdbe475babeae5e546b4038c` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:12.984660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe7d10-6e00-4ef8-820e-4cd4e51009ce None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=fd73860db589439da5178e7010d567d2, role_id=7c5539dafdbe475babeae5e546b4038c)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:12.986851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe7d10-6e00-4ef8-820e-4cd4e51009ce None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:13.141814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 611/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:11:12 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/fd73860db589439da5178e7010d567d2/roles/7c5539dafdbe475babeae5e546b4038c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:15.325428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2d701-8464-4d6b-a741-e80555129752 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.325996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2d701-8464-4d6b-a741-e80555129752 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.325996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2d701-8464-4d6b-a741-e80555129752 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.327362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 622/1233] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:15.332746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27261-4184-4b7c-b739-0c478f4951ac None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.333162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27261-4184-4b7c-b739-0c478f4951ac None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.333369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d27261-4184-4b7c-b739-0c478f4951ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.351989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-24d27261-4184-4b7c-b739-0c478f4951ac None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:15.354212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-24d27261-4184-4b7c-b739-0c478f4951ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:15.418864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 612/1234] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 21:11:15.443572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2d701-8464-4d6b-a741-e80555129752 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:15.468159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:15.541935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2d701-8464-4d6b-a741-e80555129752 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:15.577284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:15.578007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.578282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.578550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.589863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:11:15.597614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:15.598838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:15.600158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 623/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Thu May 26 21:11:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:15.605512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d27261-4184-4b7c-b739-0c478f4951ac None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:15.650917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:15.702247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d27261-4184-4b7c-b739-0c478f4951ac None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:15.737585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:15.738240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.738524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.738796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.739596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] RBAC: Adding query filter params (name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:15.739937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:15.741284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:15.749633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 613/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Thu May 26 21:11:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:15.755409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:15.838570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:15.855200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673109df-c226-4e31-873b-dc5a19aa0613 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:15.888483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:15.889307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:15.889589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:15.889866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:15.901712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:15.903298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:15.920483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:15.921588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:15.922870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:15.922870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:15.922870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:15.922870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:15.922870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 624/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:15.927609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.025915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb69f8d-9229-443e-9cd1-bc51e7c53c31 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.034528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:16.059553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.060246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.060529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.060804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.061520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:16.061828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.063183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.074805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 614/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:16.080581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.178461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b39e2-b99d-4586-bb59-722803843144 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.212873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.213664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.213941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.214265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.216924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:16.221942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.223377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:11:16.238292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:16.239242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:16.240334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 625/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:16.245969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.345822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d258910-9a62-4629-9f74-4ec64baa02fd None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.379650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.380331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.380612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.380881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.381699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:16.382004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.383403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.402283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 615/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:16.408069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.508300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48aedd-84f1-4388-8f2c-a7d9b7975b14 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.542063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.542706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.544342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.544342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.605771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:16.612171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.613675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.710527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 626/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:11:16.716959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.743462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:16.812519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e2caa-1d9f-446c-9809-37a2bc0bcf7c None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:16.845507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:16.846293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:16.846575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:16.846847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:16.856590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:16.858063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:16.875671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:16.876804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:16.878086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:16.878086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:16.878086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:16.878086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:16.878086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 616/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:16.882733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:16.981078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:16.986835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45a2ba0-75ef-44f7-b900-69e4f370d651 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.021111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.021770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.022055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.022334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.023052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:17.023365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.024837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.036449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 627/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:17.042576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.122253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:17.143356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb04ac-f35c-4195-8ce3-eee6a5d6c4bb None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.175730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.176561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.176862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.177121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.185273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.186732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:11:17.203496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:17.204740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:17.211078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 617/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:17.211714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.328636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82f313-0a95-4b65-8e03-3bb4d7631a11 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.366588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8851b6e6-42b7-4f3b-9093-9e9624d01db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.367269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851b6e6-42b7-4f3b-9093-9e9624d01db8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.367551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851b6e6-42b7-4f3b-9093-9e9624d01db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.367911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851b6e6-42b7-4f3b-9093-9e9624d01db8 None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.368723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851b6e6-42b7-4f3b-9093-9e9624d01db8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:17.369066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851b6e6-42b7-4f3b-9093-9e9624d01db8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.370555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8851b6e6-42b7-4f3b-9093-9e9624d01db8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.390665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 628/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:11:17.391400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:17.498649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8e014-b84c-40b3-b4e9-4c3a6c1fb344 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:17.535693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77a8ae1-33b3-47d9-a1f3-5ec09e66fa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:17.539609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a8ae1-33b3-47d9-a1f3-5ec09e66fa4f None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:17.539609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a8ae1-33b3-47d9-a1f3-5ec09e66fa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:17.539609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77a8ae1-33b3-47d9-a1f3-5ec09e66fa4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:17.539609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a8ae1-33b3-47d9-a1f3-5ec09e66fa4f None admin] RBAC: Adding query filter params (role.id=7c5539dafdbe475babeae5e546b4038c, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=fd73860db589439da5178e7010d567d2) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:17.539609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a8ae1-33b3-47d9-a1f3-5ec09e66fa4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:17.539609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77a8ae1-33b3-47d9-a1f3-5ec09e66fa4f None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:17.546247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 618/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:11:17 2022] GET /identity/v3/role_assignments?user.id=fd73860db589439da5178e7010d567d2&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=7c5539dafdbe475babeae5e546b4038c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:19.846511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ddc9-3733-4133-993f-a9226ff443ab None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.847074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ddc9-3733-4133-993f-a9226ff443ab None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.847074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51ddc9-3733-4133-993f-a9226ff443ab None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.848459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 629/1247] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:19.854108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130b64d-0ec4-431a-bdd9-aee8592905b1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:19.854501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130b64d-0ec4-431a-bdd9-aee8592905b1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:19.854668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130b64d-0ec4-431a-bdd9-aee8592905b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:19.859734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:19.873401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7130b64d-0ec4-431a-bdd9-aee8592905b1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:19.874613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7130b64d-0ec4-431a-bdd9-aee8592905b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:19.941490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 619/1248] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:19.964579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ddc9-3733-4133-993f-a9226ff443ab None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:19.982733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:20.064887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51ddc9-3733-4133-993f-a9226ff443ab None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.099254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.100080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.100360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.100627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.111012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:11:20.118708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:20.119688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:20.120650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 630/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:11:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:20.125920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130b64d-0ec4-431a-bdd9-aee8592905b1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.222090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130b64d-0ec4-431a-bdd9-aee8592905b1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.251305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:20.256306 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.256971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.257253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.257523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.258262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:20.258564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:20.259985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.268054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 620/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:20.273840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.351778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:20.375114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322f0bbd-1a46-460e-a8b0-ec327493c812 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.408926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.409722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.410010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.410289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.421317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:20.422792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:20.440501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:20.441499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:20.442409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:20.442409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:20.442409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:20.442409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:20.442409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 631/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:20.448772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.549068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6e231c-3c96-40af-bacd-6d7c05c6d7ce None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.587234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.588541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.588822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.589094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.589815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:20.590121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:20.591526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.604412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 621/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:20.610065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.717813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b5b12f-1c63-4719-9523-a7b947f87dd8 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.744464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:20.752535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.753333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.753613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.753882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.761578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:20.763112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:11:20.777761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:20.778915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:20.784333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 632/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:20.785416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:20.841634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:20.885689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ccc7d-397f-4d3f-b50c-cf2e8cfcbf97 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:20.919899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:20.920572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:20.920858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:20.921136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:20.921971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:20.922282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:20.923706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:20.941039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 622/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:11:20.947207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.049841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda9421-fca2-4a4d-80fa-19cabf53326b None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.084387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.085043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.085331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.085608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.139917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:21.151968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.153459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.252551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 633/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:11:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:11:21.258573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.359732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4bdeae-51fc-4cb1-882f-00b5fd7a48d5 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.375418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:21.398602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.399314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.399599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.399917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.410865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.412341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:21.430029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:21.431206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:21.432857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:21.432857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:21.432857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:21.432857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:21.432857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 623/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:21.437724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.524035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:21.536090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6acb2-fea2-4a2d-9f0c-961399e343f3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.570195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.570859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.571147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.571416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.572197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:21.572505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.573923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.585354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 634/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:21.591892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.691856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331cd64c-0a3b-4d77-bb62-b4d4ab3e40ce None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.727135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.727944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.728857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.729138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.736104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.741080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:11:21.751113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:21.752137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:21.753194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 624/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:11:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:21.758652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.859295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681924a-fd34-49f0-865b-51a56def21c3 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:21.894043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d2201-d6fb-483e-9451-f81ab47aa45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:21.894704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d2201-d6fb-483e-9451-f81ab47aa45d None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:21.894985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d2201-d6fb-483e-9451-f81ab47aa45d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:21.895259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d2201-d6fb-483e-9451-f81ab47aa45d None admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:21.896078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d2201-d6fb-483e-9451-f81ab47aa45d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:21.896392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d2201-d6fb-483e-9451-f81ab47aa45d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:21.897829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d2201-d6fb-483e-9451-f81ab47aa45d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:21.911188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 635/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:11:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:21.917507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:21.943320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:22.018204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3101397-6289-4f79-8073-3c4124c0efe3 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:22.053985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b006147-d1ec-4f28-bb7f-31b239a29b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:22.054664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b006147-d1ec-4f28-bb7f-31b239a29b73 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:22.054950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b006147-d1ec-4f28-bb7f-31b239a29b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:22.055227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b006147-d1ec-4f28-bb7f-31b239a29b73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:22.056109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b006147-d1ec-4f28-bb7f-31b239a29b73 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=fd73860db589439da5178e7010d567d2) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:22.056457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b006147-d1ec-4f28-bb7f-31b239a29b73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:22.058001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b006147-d1ec-4f28-bb7f-31b239a29b73 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:22.064585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 625/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:11:21 2022] GET /identity/v3/role_assignments?user.id=fd73860db589439da5178e7010d567d2&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:24.255494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01123bd3-b362-4a42-9db3-0c0e6ed7226b None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:24.256101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01123bd3-b362-4a42-9db3-0c0e6ed7226b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:24.256101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01123bd3-b362-4a42-9db3-0c0e6ed7226b None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:24.257436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 636/1261] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:24.263348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a875eab-0470-46c3-bf4b-2d3d70e88ad9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:24.263807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a875eab-0470-46c3-bf4b-2d3d70e88ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:24.263966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a875eab-0470-46c3-bf4b-2d3d70e88ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:24.281767 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3a875eab-0470-46c3-bf4b-2d3d70e88ad9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:24.282977 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3a875eab-0470-46c3-bf4b-2d3d70e88ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:24.349238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 626/1262] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:24.371717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01123bd3-b362-4a42-9db3-0c0e6ed7226b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:24.382711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:24.472774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01123bd3-b362-4a42-9db3-0c0e6ed7226b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:24.506579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:24.507328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:24.507611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:24.507920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:24.518685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:11:24.526243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:24.532384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:24.533410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 637/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:24.534266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a875eab-0470-46c3-bf4b-2d3d70e88ad9 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:24.551514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:24.630752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a875eab-0470-46c3-bf4b-2d3d70e88ad9 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:24.665288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:24.665959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:24.666242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:24.666519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:24.667258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:24.667565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:24.668947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:24.677191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 627/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:24.683486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:24.758097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:24.791469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5da9a-c548-4fd3-b843-71d30f6254f7 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:24.826135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:24.826962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:24.827245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:24.827518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:24.838447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:24.839944 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:24.856727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:24.865471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:24.866568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:24.866568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:24.866568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:24.866568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:24.866568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 638/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:24.866908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:24.946813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:24.965354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55ba09-dc0f-4cbc-95f5-701fbc0ac849 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:24.999930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:25.000614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:25.000895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:25.001175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:25.002020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:25.002337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:25.003762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:25.019262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 628/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:25.022802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:25.123585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a496c7-c87e-438d-8b98-5fd0bde50d3a None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:25.142409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:25.156495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:25.157294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:25.157583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:25.157861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:25.165521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:25.167483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:11:25.182796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:25.183975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:25.185300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 639/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:25.189853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:25.294996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e36d18d-dd19-4f13-81a5-e318af5d493b None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:25.329145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:25.329814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:25.330099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:25.330376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:25.331210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:25.331524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:25.333046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:25.342925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:25.352043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 629/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Thu May 26 21:11:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:25.358319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:25.457978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08416dcd-d610-4295-8647-b4998459fecb None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:25.493232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:25.494026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:25.494306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:25.494634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:25.505060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:25.506510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:25.522963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:25.524711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:25.526432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:25.526432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:25.526432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:25.526432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:25.526432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 640/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:25.530458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:25.633796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a270b49e-a97e-41d7-ba08-fecc7536c985 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:25.668296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:25.668961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:25.669243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:25.669515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:25.670225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:25.670672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:25.671959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:25.683332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 630/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:25.689316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:25.731288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:25.802791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66227a9-f860-4230-8676-4b0dc78e65bf None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:25.836420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:25.837295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:25.837671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:25.838037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:25.844772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:25.846371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:11:25.860881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:25.861970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:25.863205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 641/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:25.867890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:25.926849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:25.971338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982f85d-298b-4b0a-acc5-dadb05e8bd52 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:26.005604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a693da-b6e9-4d09-8817-ec68d87a2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:26.006268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a693da-b6e9-4d09-8817-ec68d87a2a94 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:26.006555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a693da-b6e9-4d09-8817-ec68d87a2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:26.006834 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a693da-b6e9-4d09-8817-ec68d87a2a94 None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:26.007623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a693da-b6e9-4d09-8817-ec68d87a2a94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:26.007961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a693da-b6e9-4d09-8817-ec68d87a2a94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:26.009385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a693da-b6e9-4d09-8817-ec68d87a2a94 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:26.021239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 631/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:26.027692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:26.118753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:26.127117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440a4f2-1074-4244-9bc1-5e897cecb06f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:26.160173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4ea23-a9a0-4698-9886-9304ed63857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:26.161090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4ea23-a9a0-4698-9886-9304ed63857d None admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:26.161371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4ea23-a9a0-4698-9886-9304ed63857d None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:26.161675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4ea23-a9a0-4698-9886-9304ed63857d None admin] PATH_INFO: `/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/fd73860db589439da5178e7010d567d2/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:26.163736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4ea23-a9a0-4698-9886-9304ed63857d None admin] RBAC: Authorizing `identity:create_grant(project_id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user_id=fd73860db589439da5178e7010d567d2, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:26.168033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4ea23-a9a0-4698-9886-9304ed63857d None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:26.312481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 642/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Thu May 26 21:11:26 2022] PUT /identity/v3/projects/2c2f571f2f8f4a82978ad8a99e7a9bf3/users/fd73860db589439da5178e7010d567d2/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:11:28.519940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270170-e17d-4b12-894d-33842f52947e None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:28.520530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270170-e17d-4b12-894d-33842f52947e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:28.520687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96270170-e17d-4b12-894d-33842f52947e None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:28.522477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 632/1274] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:28.528332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95df2b-15a4-49e1-a23f-404edabd5602 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:28.534638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95df2b-15a4-49e1-a23f-404edabd5602 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:28.534638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95df2b-15a4-49e1-a23f-404edabd5602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:28.545588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-2d95df2b-15a4-49e1-a23f-404edabd5602 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:28.546840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2d95df2b-15a4-49e1-a23f-404edabd5602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:28.626562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 643/1275] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:28.650279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270170-e17d-4b12-894d-33842f52947e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:28.754734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96270170-e17d-4b12-894d-33842f52947e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:28.774934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:28.793564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:28.796550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:28.796550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:28.796550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:28.805364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 21:11:28.813676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:28.814815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:28.815678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 633/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Thu May 26 21:11:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:28.821358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95df2b-15a4-49e1-a23f-404edabd5602 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:28.903236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:28.921553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95df2b-15a4-49e1-a23f-404edabd5602 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:28.955548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:28.956268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:28.956556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:28.956835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:28.957587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:28.957893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:28.959283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:28.968727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 644/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Thu May 26 21:11:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:28.974688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:29.076698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dea091-8693-4b17-889e-27b882e12cb1 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:29.113837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:29.114092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:29.114352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:29.114628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:29.125290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:29.126731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:29.146874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:29.148093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:29.149233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:29.149233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:29.149233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:29.149233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:29.149233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 634/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:29.155027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:29.265629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907405c-62f5-47ee-a23e-f7ff319942e1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:29.305481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:29.305481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:29.305481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:29.305481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:29.305979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:29.306112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:29.307501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:29.320174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 645/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:29.327677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:29.361414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:29.437015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a7988-daf9-4a80-9eae-c8420d263272 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:29.479317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:29.480165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:29.480448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:29.480723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:29.488898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:29.490375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 21:11:29.509194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:29.510979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:29.512133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 635/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:29.516904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:29.520978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:29.617471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f6acb-4cd7-45b2-9352-28d92c2e2d3c None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:29.656915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:29.657586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:29.657911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:29.658312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:29.659151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:29.659462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:29.660932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:29.679331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 646/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Thu May 26 21:11:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:29.686879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:29.787861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:29.797152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250363a-0b48-49ec-a447-1cb96f2ffe7a None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:29.835752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:29.836419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:29.836702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:29.836978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:29.905027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:29.906387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:30.002391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:30.013188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 636/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Thu May 26 21:11:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 21:11:30.019180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:30.082206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:30.120436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b262508-4d37-46d8-9819-6fa5485d0970 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:30.156459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:30.157175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:30.157446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:30.157718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:30.173689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:30.175166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 21:11:30.193683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:30.194779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:30.195934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 21:11:30.195934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 21:11:30.195934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 21:11:30.195934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:30.195934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 647/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:30.200619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:30.297075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db8887-f268-45ba-9a5e-7bb414023de2 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:30.330348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:30.330996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:30.331280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:30.331555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:30.332302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:30.332612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:30.334046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:30.345691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 637/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Thu May 26 21:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:30.351477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:30.457789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144f05a-ecf1-4b6b-8db7-c601647c22d1 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:30.471787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:30.494638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:30.495472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:30.495763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:30.496045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:30.503149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:30.504650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 21:11:30.519422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application with Lock( May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:30.530554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: ERROR keystone.server.flask.application May 26 21:11:30.532732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 648/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Thu May 26 21:11:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:30.533715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:30.584705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:30.633680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9392c06-d4a1-4f7f-bdeb-f9d6e3565992 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:30.666815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915cfd7-574a-4711-bff2-bb576e86bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:30.667465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915cfd7-574a-4711-bff2-bb576e86bd0b None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:30.667774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915cfd7-574a-4711-bff2-bb576e86bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:30.668042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915cfd7-574a-4711-bff2-bb576e86bd0b None admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:30.668816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915cfd7-574a-4711-bff2-bb576e86bd0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:30.669123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915cfd7-574a-4711-bff2-bb576e86bd0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:30.670525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915cfd7-574a-4711-bff2-bb576e86bd0b None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:30.682101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 638/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Thu May 26 21:11:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:30.688265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:30.796578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e7e5f-1202-495d-b8a2-2c52d411fd47 None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:30.833169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ddbd0-8dff-4e1a-b34c-baf9d531ec38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:30.833863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ddbd0-8dff-4e1a-b34c-baf9d531ec38 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:30.834129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ddbd0-8dff-4e1a-b34c-baf9d531ec38 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:30.834408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ddbd0-8dff-4e1a-b34c-baf9d531ec38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:30.835258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ddbd0-8dff-4e1a-b34c-baf9d531ec38 None admin] RBAC: Adding query filter params (role.id=0515c8db0fd849e5a6fdd531600c97db, scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3, user.id=fd73860db589439da5178e7010d567d2) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:30.835578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ddbd0-8dff-4e1a-b34c-baf9d531ec38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:30.837120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ddbd0-8dff-4e1a-b34c-baf9d531ec38 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:30.844711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 649/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Thu May 26 21:11:30 2022] GET /identity/v3/role_assignments?user.id=fd73860db589439da5178e7010d567d2&scope.project.id=2c2f571f2f8f4a82978ad8a99e7a9bf3&role.id=0515c8db0fd849e5a6fdd531600c97db => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:33.122806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f09f8-08a8-4b4d-bb95-dd434f0155ca None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:33.123323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f09f8-08a8-4b4d-bb95-dd434f0155ca None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:33.123323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f09f8-08a8-4b4d-bb95-dd434f0155ca None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:33.124671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 639/1288] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:11:33.130172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24de4-7151-4d85-b3c3-d48570a22b21 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:33.130603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24de4-7151-4d85-b3c3-d48570a22b21 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:33.130723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24de4-7151-4d85-b3c3-d48570a22b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:33.148272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-84a24de4-7151-4d85-b3c3-d48570a22b21 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:33.149464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-84a24de4-7151-4d85-b3c3-d48570a22b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:33.181237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:33.221152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 650/1289] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:33.243064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f09f8-08a8-4b4d-bb95-dd434f0155ca None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:33.365713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f09f8-08a8-4b4d-bb95-dd434f0155ca None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:33.382607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:33.408974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:33.409778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:33.410064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:33.410342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:33.411002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:33.412454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:33.424630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:33.425978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:33.426854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 640/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:11:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:33.432135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24de4-7151-4d85-b3c3-d48570a22b21 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:33.532186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24de4-7151-4d85-b3c3-d48570a22b21 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:33.565948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d403a1-7d37-4efc-943a-a631b172cac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:33.566653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d403a1-7d37-4efc-943a-a631b172cac7 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:33.566931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d403a1-7d37-4efc-943a-a631b172cac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:33.567208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d403a1-7d37-4efc-943a-a631b172cac7 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:33.568211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d403a1-7d37-4efc-943a-a631b172cac7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:33.568517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d403a1-7d37-4efc-943a-a631b172cac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:33.569896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d403a1-7d37-4efc-943a-a631b172cac7 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:33.575771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 651/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Thu May 26 21:11:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:33.581729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:33.682803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f647881-2c39-44c0-bd0e-cb6e767e49c9 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:33.721030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12f5d97-66e6-45b8-bce6-fe868930cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:33.721720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5d97-66e6-45b8-bce6-fe868930cb17 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:33.722006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5d97-66e6-45b8-bce6-fe868930cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:33.722285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12f5d97-66e6-45b8-bce6-fe868930cb17 None admin] PATH_INFO: `/v3/services` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:33.723012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5d97-66e6-45b8-bce6-fe868930cb17 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:33.723325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5d97-66e6-45b8-bce6-fe868930cb17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:33.724750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12f5d97-66e6-45b8-bce6-fe868930cb17 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:33.729902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 641/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Thu May 26 21:11:33 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:35.972892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ea84c-b8da-4753-9f93-2fb02da2d513 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:35.973409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ea84c-b8da-4753-9f93-2fb02da2d513 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:35.973409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ea84c-b8da-4753-9f93-2fb02da2d513 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:35.974782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 652/1293] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:35.980981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1331-22c1-4105-af8c-d82cb88d25ab None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:35.981446 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1331-22c1-4105-af8c-d82cb88d25ab None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:35.981535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c1331-22c1-4105-af8c-d82cb88d25ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:36.002975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-168c1331-22c1-4105-af8c-d82cb88d25ab None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:36.004221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-168c1331-22c1-4105-af8c-d82cb88d25ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:36.034723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:36.072432 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 642/1294] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:36.095369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ea84c-b8da-4753-9f93-2fb02da2d513 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:36.127109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:36.203438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ea84c-b8da-4753-9f93-2fb02da2d513 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:36.239785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad9f38-065d-4401-aa7d-9f534d20a273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:36.240367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad9f38-065d-4401-aa7d-9f534d20a273 None admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:36.240651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad9f38-065d-4401-aa7d-9f534d20a273 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:36.240929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad9f38-065d-4401-aa7d-9f534d20a273 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:36.241670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad9f38-065d-4401-aa7d-9f534d20a273 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:36.243033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad9f38-065d-4401-aa7d-9f534d20a273 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:36.421175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 653/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Thu May 26 21:11:36 2022] POST /identity/v3/services => generated 246 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:38.655529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d9bf2-f395-4819-ae45-726392cabbad None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.656116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d9bf2-f395-4819-ae45-726392cabbad None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.656116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583d9bf2-f395-4819-ae45-726392cabbad None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.657473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 643/1296] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:38.662785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad4ca-5cc6-49d7-b2bb-89de494919fa None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.663195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad4ca-5cc6-49d7-b2bb-89de494919fa None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.663346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ad4ca-5cc6-49d7-b2bb-89de494919fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.680491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-289ad4ca-5cc6-49d7-b2bb-89de494919fa None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:38.682272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-289ad4ca-5cc6-49d7-b2bb-89de494919fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:38.754516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 654/1297] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:38.777581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d9bf2-f395-4819-ae45-726392cabbad None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.878257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583d9bf2-f395-4819-ae45-726392cabbad None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:38.912098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:38.912816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:38.913096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:38.913366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:38.914015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:38.915986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:38.927863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:38.929042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:38.930066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 644/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:11:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:38.935308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad4ca-5cc6-49d7-b2bb-89de494919fa None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:38.953730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:39.032648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ad4ca-5cc6-49d7-b2bb-89de494919fa None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:39.066210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:39.066869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:39.067154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:39.067428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:39.068440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:39.068730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:39.070109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:39.075233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 655/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Thu May 26 21:11:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:39.081300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:39.113526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:39.179659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795a7cf-89c2-48fa-835a-14fdc0d57467 None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:39.214456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688d9c9-6a6d-406e-bac5-c6b75a8498c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:39.215115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688d9c9-6a6d-406e-bac5-c6b75a8498c2 None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:39.215393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688d9c9-6a6d-406e-bac5-c6b75a8498c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:39.215663 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688d9c9-6a6d-406e-bac5-c6b75a8498c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:39.216488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688d9c9-6a6d-406e-bac5-c6b75a8498c2 None admin] RBAC: Adding query filter params (interface=public, service_id=9c402c8ac2fb44e6864557f2372ef033, region_id=RegionOne) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:39.216800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688d9c9-6a6d-406e-bac5-c6b75a8498c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:39.218177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688d9c9-6a6d-406e-bac5-c6b75a8498c2 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:39.223897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 645/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Thu May 26 21:11:39 2022] GET /identity/v3/endpoints?service_id=9c402c8ac2fb44e6864557f2372ef033&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:39.229631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:39.310218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:39.328684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df360d77-5212-4e2f-b6a9-4d6f95adce1f None admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:39.361782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c238e176-38a0-4ee9-90f9-d84d4c0c3a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:39.362443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238e176-38a0-4ee9-90f9-d84d4c0c3a79 None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:39.362723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238e176-38a0-4ee9-90f9-d84d4c0c3a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:39.362995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238e176-38a0-4ee9-90f9-d84d4c0c3a79 None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:39.363671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238e176-38a0-4ee9-90f9-d84d4c0c3a79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:39.365121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c238e176-38a0-4ee9-90f9-d84d4c0c3a79 None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:39.372416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 656/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Thu May 26 21:11:39 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:11:41.652251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ff305-0461-4af0-b2fd-fa1189268a3e None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:41.652809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ff305-0461-4af0-b2fd-fa1189268a3e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:41.652809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ff305-0461-4af0-b2fd-fa1189268a3e None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:41.654169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 646/1302] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:11:41.659468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50ce12-801c-4cb4-ad6a-029fcda22497 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:41.659910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50ce12-801c-4cb4-ad6a-029fcda22497 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:41.660057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50ce12-801c-4cb4-ad6a-029fcda22497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:41.676807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-af50ce12-801c-4cb4-ad6a-029fcda22497 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:11:41.680133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-af50ce12-801c-4cb4-ad6a-029fcda22497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:11:41.746763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 657/1303] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:11:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 21:11:41.769503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ff305-0461-4af0-b2fd-fa1189268a3e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:41.866294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:41.872946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ff305-0461-4af0-b2fd-fa1189268a3e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:41.909240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:41.909976 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:41.910240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:41.910510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] PATH_INFO: `/v3/services/placement` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:41.911209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:41.912664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 21:11:41.925701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 21:11:41.927570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.get_or_create( May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application with Lock( May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self._enter() May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application return self.creator() May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application created_value = creator( May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: ERROR keystone.server.flask.application May 26 21:11:41.929221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 647/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Thu May 26 21:11:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 21:11:41.933661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50ce12-801c-4cb4-ad6a-029fcda22497 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.022317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:42.045483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50ce12-801c-4cb4-ad6a-029fcda22497 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.079268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080cd8a-20a5-4ef2-98f8-7782f4632f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.079960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080cd8a-20a5-4ef2-98f8-7782f4632f1c None admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.080240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080cd8a-20a5-4ef2-98f8-7782f4632f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.080510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080cd8a-20a5-4ef2-98f8-7782f4632f1c None admin] PATH_INFO: `/v3/services` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.081213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080cd8a-20a5-4ef2-98f8-7782f4632f1c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:11:42.081517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080cd8a-20a5-4ef2-98f8-7782f4632f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.082899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080cd8a-20a5-4ef2-98f8-7782f4632f1c None admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.088522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 658/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Thu May 26 21:11:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:11:42.094422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:11:42.194702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fec0f9-8127-4e9a-a5c7-811c8cc804ab None admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:11:42.228679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a3847-df19-4c19-b737-ded2ea2b7358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:11:42.229333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a3847-df19-4c19-b737-ded2ea2b7358 None admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:11:42.229617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a3847-df19-4c19-b737-ded2ea2b7358 None admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:11:42.229893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a3847-df19-4c19-b737-ded2ea2b7358 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:11:42.230632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a3847-df19-4c19-b737-ded2ea2b7358 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:11:42.232049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a3847-df19-4c19-b737-ded2ea2b7358 None admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:11:42.317133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 648/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Thu May 26 21:11:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:11:52.117507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=30 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:11:52.312394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:54.016906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504373-7b92-49a7-abaf-576ef5279240 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:54.017534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504373-7b92-49a7-abaf-576ef5279240 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:54.017732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d504373-7b92-49a7-abaf-576ef5279240 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:54.025038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734210a-5c1a-4eea-b8a3-18baab0539c0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:54.025576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734210a-5c1a-4eea-b8a3-18baab0539c0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:54.025828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734210a-5c1a-4eea-b8a3-18baab0539c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:54.027827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 659/1307] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:17:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:17:54.125816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c734210a-5c1a-4eea-b8a3-18baab0539c0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:54.127268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c734210a-5c1a-4eea-b8a3-18baab0539c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:54.284504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 649/1308] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:17:54.326118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe8cf-accc-4613-93eb-e336d1d16ffd None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:54.326446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe8cf-accc-4613-93eb-e336d1d16ffd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:54.326720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfe8cf-accc-4613-93eb-e336d1d16ffd None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:54.328376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 660/1309] 10.222.0.42 () {54 vars in 964 bytes} [Thu May 26 21:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:17:54.335040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7600365-280f-4afb-9914-f5a63a560c21 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:54.335268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7600365-280f-4afb-9914-f5a63a560c21 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:54.335605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7600365-280f-4afb-9914-f5a63a560c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:54.446357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b7600365-280f-4afb-9914-f5a63a560c21 None None] MFA Rules not processed for user `e757ff94b7474fd99a1c7221be19c64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:54.456898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b7600365-280f-4afb-9914-f5a63a560c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:54.502684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:54.597255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfe8cf-accc-4613-93eb-e336d1d16ffd None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:54.599242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fcbfe8cf-accc-4613-93eb-e336d1d16ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:54.604662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 650/1310] 10.222.0.42 () {58 vars in 1061 bytes} [Thu May 26 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:17:54.742579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfe8cf-accc-4613-93eb-e336d1d16ffd None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:54.838141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b88cec-072e-47c9-b49e-165cf7c75c05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:54.839183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b88cec-072e-47c9-b49e-165cf7c75c05 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:54.839684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b88cec-072e-47c9-b49e-165cf7c75c05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:54.840282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b88cec-072e-47c9-b49e-165cf7c75c05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:54.842226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-14b88cec-072e-47c9-b49e-165cf7c75c05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:54.899586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:54.959753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b88cec-072e-47c9-b49e-165cf7c75c05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:54.962037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b88cec-072e-47c9-b49e-165cf7c75c05 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:54.981887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 661/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:17:57.554521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dde6f-f36d-4e49-8cd2-fdca100c166d None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:57.555071 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dde6f-f36d-4e49-8cd2-fdca100c166d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:57.555071 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66dde6f-f36d-4e49-8cd2-fdca100c166d None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:57.556513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 651/1312] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:17:57.562147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d59a2-64df-4839-85cb-4f0ad59dbf0a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:57.563270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d59a2-64df-4839-85cb-4f0ad59dbf0a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:57.563792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d59a2-64df-4839-85cb-4f0ad59dbf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:57.655538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-487d59a2-64df-4839-85cb-4f0ad59dbf0a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:17:57.657165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-487d59a2-64df-4839-85cb-4f0ad59dbf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:57.770004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 662/1313] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:17:57.808387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dde6f-f36d-4e49-8cd2-fdca100c166d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:17:57.811372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d66dde6f-f36d-4e49-8cd2-fdca100c166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:57.925412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66dde6f-f36d-4e49-8cd2-fdca100c166d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:17:57.997002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:17:57.999643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9bdde2-af0a-4940-b82f-fffb749d872b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:17:58.000342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bdde2-af0a-4940-b82f-fffb749d872b service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:17:58.000626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bdde2-af0a-4940-b82f-fffb749d872b service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:17:58.000906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9bdde2-af0a-4940-b82f-fffb749d872b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:17:58.002576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9b9bdde2-af0a-4940-b82f-fffb749d872b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:17:58.113003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bdde2-af0a-4940-b82f-fffb749d872b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:17:58.114437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9bdde2-af0a-4940-b82f-fffb749d872b service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:17:58.133194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 652/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:00.818034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9c374-07ef-4bf7-b9e6-a8e9967c0b6e None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:00.818572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9c374-07ef-4bf7-b9e6-a8e9967c0b6e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:00.818572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9c374-07ef-4bf7-b9e6-a8e9967c0b6e None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:00.820004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 663/1315] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:00.825320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431d8c-99ea-48e2-b990-767891ba6d95 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:00.825726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431d8c-99ea-48e2-b990-767891ba6d95 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:00.825880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5431d8c-99ea-48e2-b990-767891ba6d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:00.888695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d5431d8c-99ea-48e2-b990-767891ba6d95 None None] MFA Rules not processed for user `591c45f7bb064291b85f4b4a3998abe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:00.889900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d5431d8c-99ea-48e2-b990-767891ba6d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:00.941101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:00.985521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 653/1316] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:01.024668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9c374-07ef-4bf7-b9e6-a8e9967c0b6e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:01.026220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a4b9c374-07ef-4bf7-b9e6-a8e9967c0b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:01.036250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:01.130012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9c374-07ef-4bf7-b9e6-a8e9967c0b6e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:01.201389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225aaff-de2d-4987-91f7-199965812b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:01.202031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225aaff-de2d-4987-91f7-199965812b15 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:01.202315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225aaff-de2d-4987-91f7-199965812b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:01.202589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225aaff-de2d-4987-91f7-199965812b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:01.204279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8225aaff-de2d-4987-91f7-199965812b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:01.264621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:01.310517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225aaff-de2d-4987-91f7-199965812b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:01.311991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225aaff-de2d-4987-91f7-199965812b15 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:01.329793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 664/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:01 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:04.326264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b590c9-d7e5-49d2-b1c2-069662cfdb83 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:04.326804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b590c9-d7e5-49d2-b1c2-069662cfdb83 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:04.326930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b590c9-d7e5-49d2-b1c2-069662cfdb83 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:04.328227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 654/1318] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:04.333325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f51c42-b0c4-495d-be8b-0a0d2c2a9b90 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:04.333732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f51c42-b0c4-495d-be8b-0a0d2c2a9b90 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:04.333880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f51c42-b0c4-495d-be8b-0a0d2c2a9b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:04.397488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-78f51c42-b0c4-495d-be8b-0a0d2c2a9b90 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:04.399164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-78f51c42-b0c4-495d-be8b-0a0d2c2a9b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:04.493434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 665/1319] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:04.544498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518abb0-b030-4cc6-a95f-d5ff38092be0 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:04.544786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518abb0-b030-4cc6-a95f-d5ff38092be0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:04.545068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518abb0-b030-4cc6-a95f-d5ff38092be0 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:04.546458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 655/1320] 10.222.0.42 () {54 vars in 964 bytes} [Thu May 26 21:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:04.552008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424e3be-d84b-4fd2-9310-063a5ee15f38 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:04.552300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424e3be-d84b-4fd2-9310-063a5ee15f38 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:04.552593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424e3be-d84b-4fd2-9310-063a5ee15f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:04.574399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:04.627401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6424e3be-d84b-4fd2-9310-063a5ee15f38 None None] MFA Rules not processed for user `e757ff94b7474fd99a1c7221be19c64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:04.628648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6424e3be-d84b-4fd2-9310-063a5ee15f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:04.703212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 666/1321] 10.222.0.42 () {58 vars in 1061 bytes} [Thu May 26 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:04.712790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518abb0-b030-4cc6-a95f-d5ff38092be0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:04.714358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2518abb0-b030-4cc6-a95f-d5ff38092be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:04.816406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518abb0-b030-4cc6-a95f-d5ff38092be0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:04.888576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dec2bd-a6ed-4eb7-a718-229f82a80913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:04.889219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dec2bd-a6ed-4eb7-a718-229f82a80913 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:04.889499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dec2bd-a6ed-4eb7-a718-229f82a80913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:04.889771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dec2bd-a6ed-4eb7-a718-229f82a80913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:04.891427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-85dec2bd-a6ed-4eb7-a718-229f82a80913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:04.899188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:04.999017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dec2bd-a6ed-4eb7-a718-229f82a80913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:05.000407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dec2bd-a6ed-4eb7-a718-229f82a80913 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:05.019226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 656/1322] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:08.684199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233b4b0-9e13-402a-8d2a-25309e5f9e9c None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:08.684747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233b4b0-9e13-402a-8d2a-25309e5f9e9c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:08.684747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233b4b0-9e13-402a-8d2a-25309e5f9e9c None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:08.686142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 667/1323] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:08.691800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafdd31-e335-4eb2-a716-722a93098230 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:08.692192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafdd31-e335-4eb2-a716-722a93098230 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:08.692382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aafdd31-e335-4eb2-a716-722a93098230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:08.759027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7aafdd31-e335-4eb2-a716-722a93098230 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:08.760265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7aafdd31-e335-4eb2-a716-722a93098230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:08.818178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:08.858981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 657/1324] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:08.891688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233b4b0-9e13-402a-8d2a-25309e5f9e9c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:08.893277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a233b4b0-9e13-402a-8d2a-25309e5f9e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:09.001651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233b4b0-9e13-402a-8d2a-25309e5f9e9c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:09.014295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:09.076874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8941f22-f739-4286-939b-c23db13fae6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:09.077518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941f22-f739-4286-939b-c23db13fae6c service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:09.077798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941f22-f739-4286-939b-c23db13fae6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:09.078071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8941f22-f739-4286-939b-c23db13fae6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:09.079705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d8941f22-f739-4286-939b-c23db13fae6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:09.186115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8941f22-f739-4286-939b-c23db13fae6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:09.187845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8941f22-f739-4286-939b-c23db13fae6c service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:09.206256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 668/1325] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:11.469390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c497a-9cd4-4a94-9a4b-1d7d084eed48 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:11.469989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c497a-9cd4-4a94-9a4b-1d7d084eed48 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:11.469989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775c497a-9cd4-4a94-9a4b-1d7d084eed48 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:11.471344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 658/1326] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:11.477135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460c566-40f0-4448-a50d-0d3e35ce688e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:11.477810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460c566-40f0-4448-a50d-0d3e35ce688e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:11.477810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6460c566-40f0-4448-a50d-0d3e35ce688e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:11.525223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:11.543083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6460c566-40f0-4448-a50d-0d3e35ce688e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:11.544340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6460c566-40f0-4448-a50d-0d3e35ce688e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:11.647705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 669/1327] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:11.684002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c497a-9cd4-4a94-9a4b-1d7d084eed48 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:11.685565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-775c497a-9cd4-4a94-9a4b-1d7d084eed48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:11.790872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775c497a-9cd4-4a94-9a4b-1d7d084eed48 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:11.865918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e96a3e-6b21-4cd9-8d3d-d032212c2914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:11.867076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e96a3e-6b21-4cd9-8d3d-d032212c2914 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:11.867358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e96a3e-6b21-4cd9-8d3d-d032212c2914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:11.867633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e96a3e-6b21-4cd9-8d3d-d032212c2914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:11.869305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c5e96a3e-6b21-4cd9-8d3d-d032212c2914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:11.887064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:11.980117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e96a3e-6b21-4cd9-8d3d-d032212c2914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:11.981556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e96a3e-6b21-4cd9-8d3d-d032212c2914 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:12.000269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 659/1328] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:14.319528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18768cee-9002-4a94-921b-929f85fd0d61 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:14.320445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18768cee-9002-4a94-921b-929f85fd0d61 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:14.320960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18768cee-9002-4a94-921b-929f85fd0d61 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:14.322654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 670/1329] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:14.328231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4705a-757e-4edb-b245-55d78cb92678 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:14.328715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4705a-757e-4edb-b245-55d78cb92678 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:14.328810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc4705a-757e-4edb-b245-55d78cb92678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:14.396765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-dbc4705a-757e-4edb-b245-55d78cb92678 None None] MFA Rules not processed for user `591c45f7bb064291b85f4b4a3998abe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:14.398034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dbc4705a-757e-4edb-b245-55d78cb92678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:14.465333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:14.499194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 660/1330] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:14.535852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18768cee-9002-4a94-921b-929f85fd0d61 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:14.537497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-18768cee-9002-4a94-921b-929f85fd0d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:14.649642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18768cee-9002-4a94-921b-929f85fd0d61 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:14.671184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:14.729793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1c1bc-4053-44b6-ac95-0f8187d235c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:14.730449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1c1bc-4053-44b6-ac95-0f8187d235c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:14.730734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1c1bc-4053-44b6-ac95-0f8187d235c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:14.731009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1c1bc-4053-44b6-ac95-0f8187d235c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:14.732710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b6c1c1bc-4053-44b6-ac95-0f8187d235c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:14.844844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1c1bc-4053-44b6-ac95-0f8187d235c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:14.846989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1c1bc-4053-44b6-ac95-0f8187d235c3 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:14.866920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 671/1331] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:17.612167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4e18c-cbb5-4f41-8be3-d42d04939c1d None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:17.612675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4e18c-cbb5-4f41-8be3-d42d04939c1d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:17.612783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4e18c-cbb5-4f41-8be3-d42d04939c1d None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:17.614235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 661/1332] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:17.620524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c3a9a-e588-4556-ac22-8ba39417e315 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:17.621003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c3a9a-e588-4556-ac22-8ba39417e315 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:17.621431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c3a9a-e588-4556-ac22-8ba39417e315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:17.697420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:17.700403 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3c8c3a9a-e588-4556-ac22-8ba39417e315 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:17.701647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3c8c3a9a-e588-4556-ac22-8ba39417e315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:17.803197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 672/1333] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:17.841653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4e18c-cbb5-4f41-8be3-d42d04939c1d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:17.843197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a4f4e18c-cbb5-4f41-8be3-d42d04939c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:17.963992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4e18c-cbb5-4f41-8be3-d42d04939c1d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:18.049969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02db660-ed77-4200-abc9-bbcb9a9e5bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:18.050637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02db660-ed77-4200-abc9-bbcb9a9e5bae service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:18.050924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02db660-ed77-4200-abc9-bbcb9a9e5bae service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:18.051209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02db660-ed77-4200-abc9-bbcb9a9e5bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:18.052961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e02db660-ed77-4200-abc9-bbcb9a9e5bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:18.082535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:18.204881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02db660-ed77-4200-abc9-bbcb9a9e5bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:18.206541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02db660-ed77-4200-abc9-bbcb9a9e5bae service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:18.227309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 662/1334] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:20.504918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9d2e2-5df5-4a8a-99d5-8b9007f87138 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:20.505461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9d2e2-5df5-4a8a-99d5-8b9007f87138 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:20.505587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9d2e2-5df5-4a8a-99d5-8b9007f87138 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:20.506881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 673/1335] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:20.512359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5c98e-b48b-402b-b9fa-8117ad4a1e98 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:20.512800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5c98e-b48b-402b-b9fa-8117ad4a1e98 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:20.512897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5c98e-b48b-402b-b9fa-8117ad4a1e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:20.576202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-fba5c98e-b48b-402b-b9fa-8117ad4a1e98 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:20.577398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-fba5c98e-b48b-402b-b9fa-8117ad4a1e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:20.655966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:20.678418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 663/1336] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:20.717804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9d2e2-5df5-4a8a-99d5-8b9007f87138 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:20.719361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-aba9d2e2-5df5-4a8a-99d5-8b9007f87138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:20.822022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9d2e2-5df5-4a8a-99d5-8b9007f87138 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:20.849810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:20.894591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd78b0e1-3974-4b36-a6e3-485ce5d7ce94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:20.895231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78b0e1-3974-4b36-a6e3-485ce5d7ce94 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:20.895510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78b0e1-3974-4b36-a6e3-485ce5d7ce94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:20.895814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd78b0e1-3974-4b36-a6e3-485ce5d7ce94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:20.897963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-dd78b0e1-3974-4b36-a6e3-485ce5d7ce94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:21.005436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78b0e1-3974-4b36-a6e3-485ce5d7ce94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:21.007427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd78b0e1-3974-4b36-a6e3-485ce5d7ce94 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:21.029053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 674/1337] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:23.817705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9557ba0-da91-4893-b80c-3bf7d0416fba None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:23.818311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9557ba0-da91-4893-b80c-3bf7d0416fba None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:23.818311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9557ba0-da91-4893-b80c-3bf7d0416fba None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:23.820254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 664/1338] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:23.825311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75630d6-57c9-4867-9c29-4bf34281f4cf None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:23.825748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75630d6-57c9-4867-9c29-4bf34281f4cf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:23.825852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75630d6-57c9-4867-9c29-4bf34281f4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:23.888646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f75630d6-57c9-4867-9c29-4bf34281f4cf None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:23.889847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f75630d6-57c9-4867-9c29-4bf34281f4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:23.898057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:23.984440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 675/1339] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:24.020425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9557ba0-da91-4893-b80c-3bf7d0416fba None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:24.022016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d9557ba0-da91-4893-b80c-3bf7d0416fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:24.131964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9557ba0-da91-4893-b80c-3bf7d0416fba None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:24.207584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377aed9-7c98-4b3c-bc02-b9085116045a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:24.208273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377aed9-7c98-4b3c-bc02-b9085116045a service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:24.208557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377aed9-7c98-4b3c-bc02-b9085116045a service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:24.208833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377aed9-7c98-4b3c-bc02-b9085116045a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:24.210884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c377aed9-7c98-4b3c-bc02-b9085116045a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:24.248181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:24.326716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377aed9-7c98-4b3c-bc02-b9085116045a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:24.328185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377aed9-7c98-4b3c-bc02-b9085116045a service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:24.348970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 665/1340] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:28.765491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628635-3477-4f9e-9c33-bda2aa978b45 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:28.766092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628635-3477-4f9e-9c33-bda2aa978b45 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:28.766092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a628635-3477-4f9e-9c33-bda2aa978b45 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:28.767443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 676/1341] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:28.773165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f7c19-95a2-44ba-b180-bbcfbe00f990 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:28.773678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f7c19-95a2-44ba-b180-bbcfbe00f990 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:28.773786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f7c19-95a2-44ba-b180-bbcfbe00f990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:28.822271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-174f7c19-95a2-44ba-b180-bbcfbe00f990 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:28.823499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-174f7c19-95a2-44ba-b180-bbcfbe00f990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:28.902826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 666/1342] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:28.941237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628635-3477-4f9e-9c33-bda2aa978b45 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:28.943462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6a628635-3477-4f9e-9c33-bda2aa978b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:28.980243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a628635-3477-4f9e-9c33-bda2aa978b45 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:28.984219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536365e-be02-4720-b12d-6e82d33b001a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:28.984898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536365e-be02-4720-b12d-6e82d33b001a service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:28.985210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536365e-be02-4720-b12d-6e82d33b001a service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:28.985483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536365e-be02-4720-b12d-6e82d33b001a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:28.995771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536365e-be02-4720-b12d-6e82d33b001a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:29.001003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536365e-be02-4720-b12d-6e82d33b001a service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:29.007810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 677/1343] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:32.192752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4d239-b711-40ed-b245-016c98d582a4 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:32.193281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4d239-b711-40ed-b245-016c98d582a4 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:32.193377 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4d239-b711-40ed-b245-016c98d582a4 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:32.194739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 667/1344] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:32.200070 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702dbd-4c0b-4a4b-9b5f-3a1202159c85 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:32.200535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702dbd-4c0b-4a4b-9b5f-3a1202159c85 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:32.200774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a702dbd-4c0b-4a4b-9b5f-3a1202159c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:32.228942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7a702dbd-4c0b-4a4b-9b5f-3a1202159c85 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:32.230132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7a702dbd-4c0b-4a4b-9b5f-3a1202159c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:32.238014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 678/1345] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:32.275243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4d239-b711-40ed-b245-016c98d582a4 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:32.276982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c9d4d239-b711-40ed-b245-016c98d582a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:32.289842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4d239-b711-40ed-b245-016c98d582a4 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:32.293364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a5c6a-e8c7-4d95-a17d-71eb7b3de7b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:32.294004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a5c6a-e8c7-4d95-a17d-71eb7b3de7b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:32.294287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a5c6a-e8c7-4d95-a17d-71eb7b3de7b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:32.294564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a5c6a-e8c7-4d95-a17d-71eb7b3de7b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:32.303710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a5c6a-e8c7-4d95-a17d-71eb7b3de7b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:32.305161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a5c6a-e8c7-4d95-a17d-71eb7b3de7b9 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:32.314882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 668/1346] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:36.691108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3ac7-d358-4550-9cc5-1b2b2d32307f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:36.691666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3ac7-d358-4550-9cc5-1b2b2d32307f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:36.691666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b3ac7-d358-4550-9cc5-1b2b2d32307f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:36.693235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 679/1347] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:36.700105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cee07-c7c4-4b3e-b90f-c88fafd9aaa2 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:36.700585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cee07-c7c4-4b3e-b90f-c88fafd9aaa2 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:36.700675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3cee07-c7c4-4b3e-b90f-c88fafd9aaa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:36.722944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ee3cee07-c7c4-4b3e-b90f-c88fafd9aaa2 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:36.724237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ee3cee07-c7c4-4b3e-b90f-c88fafd9aaa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:36.732802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 669/1348] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:36.770140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b3ac7-d358-4550-9cc5-1b2b2d32307f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:36.781965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b3ac7-d358-4550-9cc5-1b2b2d32307f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:36.785639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e556e-badc-468f-97ff-92d485c39221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:36.786383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e556e-badc-468f-97ff-92d485c39221 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:36.786742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e556e-badc-468f-97ff-92d485c39221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:36.787090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e556e-badc-468f-97ff-92d485c39221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:36.796289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e556e-badc-468f-97ff-92d485c39221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:36.797724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e556e-badc-468f-97ff-92d485c39221 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:36.807175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 680/1349] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:42.106291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ae396-069b-4fd5-90bf-207559a349d3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:42.107261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ae396-069b-4fd5-90bf-207559a349d3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:42.107769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ae396-069b-4fd5-90bf-207559a349d3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:42.109649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 670/1350] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:42.114714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c9296-1e6b-46c4-8002-32156d16cb49 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:42.115148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c9296-1e6b-46c4-8002-32156d16cb49 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:42.115267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c9296-1e6b-46c4-8002-32156d16cb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:42.239386 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7a6c9296-1e6b-46c4-8002-32156d16cb49 None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:42.241634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7a6c9296-1e6b-46c4-8002-32156d16cb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:42.277109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:42.341641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 681/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:46.607918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f24a8-3e48-4644-ade0-1e747c180b48 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:46.608452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f24a8-3e48-4644-ade0-1e747c180b48 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:46.608452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f24a8-3e48-4644-ade0-1e747c180b48 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:46.610015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 671/1352] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:46.615363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae6dc8-abe0-4b92-a9a9-9286cc75103f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:46.615913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae6dc8-abe0-4b92-a9a9-9286cc75103f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:46.616019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae6dc8-abe0-4b92-a9a9-9286cc75103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:46.635428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-64ae6dc8-abe0-4b92-a9a9-9286cc75103f None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:46.637102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-64ae6dc8-abe0-4b92-a9a9-9286cc75103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:46.646265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 682/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:18:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:46.717394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=93 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:18:48.134419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca631ad5-19cc-4bbe-bb39-37261564a280 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:48.135007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca631ad5-19cc-4bbe-bb39-37261564a280 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:48.135007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca631ad5-19cc-4bbe-bb39-37261564a280 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:48.136378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 672/1354] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:48.141550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5e60-03f1-4926-99f9-2d7fe2afc9fe None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:48.142001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5e60-03f1-4926-99f9-2d7fe2afc9fe None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:48.142098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe5e60-03f1-4926-99f9-2d7fe2afc9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:48.160544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f4fe5e60-03f1-4926-99f9-2d7fe2afc9fe None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:48.161731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f4fe5e60-03f1-4926-99f9-2d7fe2afc9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:48.170878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 683/1355] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:52.209743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5f02f-13bc-40da-8c86-cfaa15bd9b49 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:52.210281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5f02f-13bc-40da-8c86-cfaa15bd9b49 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:52.210709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5f02f-13bc-40da-8c86-cfaa15bd9b49 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:52.212470 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 673/1356] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:52.218125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cab15-50b9-4f7e-baee-efa44930cba8 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:52.218606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cab15-50b9-4f7e-baee-efa44930cba8 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:52.218687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cab15-50b9-4f7e-baee-efa44930cba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:52.237164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-500cab15-50b9-4f7e-baee-efa44930cba8 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:52.238339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-500cab15-50b9-4f7e-baee-efa44930cba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:52.247059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 684/1357] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:18:52.304235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42cf37-94b3-470b-b48f-1f9a9f3fcd95 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:52.304743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42cf37-94b3-470b-b48f-1f9a9f3fcd95 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:52.305025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42cf37-94b3-470b-b48f-1f9a9f3fcd95 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:52.306409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 674/1358] 10.222.0.42 () {54 vars in 961 bytes} [Thu May 26 21:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:52.314868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8271d-4455-4c95-ad39-ee80b3dd0995 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:52.315469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8271d-4455-4c95-ad39-ee80b3dd0995 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:52.316011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8271d-4455-4c95-ad39-ee80b3dd0995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:52.385093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-67c8271d-4455-4c95-ad39-ee80b3dd0995 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:52.386421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-67c8271d-4455-4c95-ad39-ee80b3dd0995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:52.476416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 685/1359] 10.222.0.42 () {58 vars in 1058 bytes} [Thu May 26 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:18:52.484268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42cf37-94b3-470b-b48f-1f9a9f3fcd95 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:52.500672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42cf37-94b3-470b-b48f-1f9a9f3fcd95 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:52.507401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b49882-1c25-4c6f-9726-712f5a93b004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:52.508215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49882-1c25-4c6f-9726-712f5a93b004 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:52.508623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49882-1c25-4c6f-9726-712f5a93b004 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:52.509012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b49882-1c25-4c6f-9726-712f5a93b004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:52.518857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b49882-1c25-4c6f-9726-712f5a93b004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:52.520616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b49882-1c25-4c6f-9726-712f5a93b004 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:52.532939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 675/1360] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:54.997085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ef104-028d-47bc-92d1-8b3a081cac67 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:54.998196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ef104-028d-47bc-92d1-8b3a081cac67 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:54.998714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ef104-028d-47bc-92d1-8b3a081cac67 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:55.000388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 686/1361] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:55.006863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797794c4-7b0e-49ba-b5c8-216ec1c7b3b7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:55.007779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797794c4-7b0e-49ba-b5c8-216ec1c7b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:55.008288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797794c4-7b0e-49ba-b5c8-216ec1c7b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:55.036973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-797794c4-7b0e-49ba-b5c8-216ec1c7b3b7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:55.038418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-797794c4-7b0e-49ba-b5c8-216ec1c7b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:55.048362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 676/1362] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:55.096541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ef104-028d-47bc-92d1-8b3a081cac67 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:55.112521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ef104-028d-47bc-92d1-8b3a081cac67 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:55.119598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3e7f3-e897-4dbc-b99c-cfa351342d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:55.120634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3e7f3-e897-4dbc-b99c-cfa351342d60 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:55.121158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3e7f3-e897-4dbc-b99c-cfa351342d60 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:55.121666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3e7f3-e897-4dbc-b99c-cfa351342d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:55.135985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3e7f3-e897-4dbc-b99c-cfa351342d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:55.137317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b3e7f3-e897-4dbc-b99c-cfa351342d60 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:55.162234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 687/1363] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:18:57.905603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fc2c-054a-46a8-bd7f-e002cb784545 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:57.906113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fc2c-054a-46a8-bd7f-e002cb784545 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:57.906113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc1fc2c-054a-46a8-bd7f-e002cb784545 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:57.907419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 677/1364] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:57.912615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3165a40-11e1-4fff-b69d-c14aab23b7f9 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:57.913050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3165a40-11e1-4fff-b69d-c14aab23b7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:57.913345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3165a40-11e1-4fff-b69d-c14aab23b7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:57.931216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f3165a40-11e1-4fff-b69d-c14aab23b7f9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:57.932483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f3165a40-11e1-4fff-b69d-c14aab23b7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:57.940654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 688/1365] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:18:57.988920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd439-ce6b-4e52-830b-4dd722867f1e None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:57.989199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd439-ce6b-4e52-830b-4dd722867f1e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:57.989477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61dd439-ce6b-4e52-830b-4dd722867f1e None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:57.990900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 678/1366] 10.222.0.42 () {54 vars in 961 bytes} [Thu May 26 21:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:18:57.996654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9c5bb-602f-409d-81ab-4f6c030e1c50 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:57.996943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9c5bb-602f-409d-81ab-4f6c030e1c50 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:57.997224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe9c5bb-602f-409d-81ab-4f6c030e1c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:58.016072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3fe9c5bb-602f-409d-81ab-4f6c030e1c50 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:18:58.017388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3fe9c5bb-602f-409d-81ab-4f6c030e1c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:18:58.027867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 689/1367] 10.222.0.42 () {58 vars in 1058 bytes} [Thu May 26 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:18:58.036044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dd439-ce6b-4e52-830b-4dd722867f1e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:18:58.050842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61dd439-ce6b-4e52-830b-4dd722867f1e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:18:58.056621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf9ddaf-5dfe-41e6-b369-6f70daab552f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:18:58.057274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ddaf-5dfe-41e6-b369-6f70daab552f service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:18:58.057560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ddaf-5dfe-41e6-b369-6f70daab552f service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:18:58.057837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf9ddaf-5dfe-41e6-b369-6f70daab552f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:18:58.066935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ddaf-5dfe-41e6-b369-6f70daab552f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:18:58.069102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf9ddaf-5dfe-41e6-b369-6f70daab552f service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:18:58.079113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 679/1368] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:00.462825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ff74d-a2c7-4dae-8d2c-0156f79b2ecb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:00.463364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ff74d-a2c7-4dae-8d2c-0156f79b2ecb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:00.463364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ff74d-a2c7-4dae-8d2c-0156f79b2ecb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:00.464771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 690/1369] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:00.470208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ccddb-caf3-4280-8fe3-17535069630d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:00.470652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ccddb-caf3-4280-8fe3-17535069630d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:00.470743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ccddb-caf3-4280-8fe3-17535069630d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:00.489304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-187ccddb-caf3-4280-8fe3-17535069630d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:00.490526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-187ccddb-caf3-4280-8fe3-17535069630d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:00.498714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 680/1370] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:00.543876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ff74d-a2c7-4dae-8d2c-0156f79b2ecb None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:00.558844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc20f21-8a60-4a75-baee-147d9414087a None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:00.559136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc20f21-8a60-4a75-baee-147d9414087a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:00.559401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc20f21-8a60-4a75-baee-147d9414087a None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:00.560144 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ff74d-a2c7-4dae-8d2c-0156f79b2ecb None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:00.561067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 681/1371] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:19:00.566937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22e1d3-245d-4d0c-b009-faa0dfdb5eec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:00.567300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af42e3d-a0a3-4eaf-a04c-944963240f23 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:00.567691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af42e3d-a0a3-4eaf-a04c-944963240f23 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:00.567815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e1d3-245d-4d0c-b009-faa0dfdb5eec service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:00.568079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e1d3-245d-4d0c-b009-faa0dfdb5eec service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:00.568275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af42e3d-a0a3-4eaf-a04c-944963240f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:00.568574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e1d3-245d-4d0c-b009-faa0dfdb5eec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:00.579490 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22e1d3-245d-4d0c-b009-faa0dfdb5eec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:00.584664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22e1d3-245d-4d0c-b009-faa0dfdb5eec service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:00.588560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6af42e3d-a0a3-4eaf-a04c-944963240f23 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:00.589959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6af42e3d-a0a3-4eaf-a04c-944963240f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:00.593210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 691/1372] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:00.604151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 682/1373] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:19:00.929532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef9c9-204a-475f-a65e-3cfb86682738 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:00.929825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef9c9-204a-475f-a65e-3cfb86682738 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:00.930434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cef9c9-204a-475f-a65e-3cfb86682738 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:00.932258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 692/1374] 10.222.0.42 () {54 vars in 963 bytes} [Thu May 26 21:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:00.940551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5683f67-3512-4f97-a696-def62d140c62 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:00.940841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5683f67-3512-4f97-a696-def62d140c62 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:00.941320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5683f67-3512-4f97-a696-def62d140c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:01.002256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f5683f67-3512-4f97-a696-def62d140c62 None None] MFA Rules not processed for user `b6cf996e1fdb4a458d06bfd23d082dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:01.003688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f5683f67-3512-4f97-a696-def62d140c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:01.075976 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 683/1375] 10.222.0.42 () {58 vars in 1060 bytes} [Thu May 26 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:01.082498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef9c9-204a-475f-a65e-3cfb86682738 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:01.095537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cef9c9-204a-475f-a65e-3cfb86682738 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:01.099375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7a2f6-c831-463d-a75a-d98bed57e06c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:01.104336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a2f6-c831-463d-a75a-d98bed57e06c service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:01.104336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a2f6-c831-463d-a75a-d98bed57e06c service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:01.104336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a2f6-c831-463d-a75a-d98bed57e06c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:01.111176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7a2f6-c831-463d-a75a-d98bed57e06c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:01.112812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7a2f6-c831-463d-a75a-d98bed57e06c service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:01.117888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:19:01.125146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 693/1376] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:03.004318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d90be4-6894-4aaa-9bab-7b35206e1d1b None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:03.004904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d90be4-6894-4aaa-9bab-7b35206e1d1b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:03.004904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d90be4-6894-4aaa-9bab-7b35206e1d1b None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:03.006792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 684/1377] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:03.012900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ff00a-62a8-47b2-904a-55a68357f43e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:03.013492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ff00a-62a8-47b2-904a-55a68357f43e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:03.013492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ff00a-62a8-47b2-904a-55a68357f43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:03.033373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0b8ff00a-62a8-47b2-904a-55a68357f43e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:03.034629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0b8ff00a-62a8-47b2-904a-55a68357f43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:03.042847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 694/1378] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:03.090153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d90be4-6894-4aaa-9bab-7b35206e1d1b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:03.105699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d90be4-6894-4aaa-9bab-7b35206e1d1b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:03.111184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884b23a7-3fd7-4e7f-a3a0-52b229b99f51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:03.111961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b23a7-3fd7-4e7f-a3a0-52b229b99f51 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:03.112144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b23a7-3fd7-4e7f-a3a0-52b229b99f51 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:03.112425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884b23a7-3fd7-4e7f-a3a0-52b229b99f51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:03.121630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b23a7-3fd7-4e7f-a3a0-52b229b99f51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:03.123218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884b23a7-3fd7-4e7f-a3a0-52b229b99f51 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:03.134130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 685/1379] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:05.527631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196816b-9f85-48ca-b252-62ee220ba9dc None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:05.528252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196816b-9f85-48ca-b252-62ee220ba9dc None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:05.528252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196816b-9f85-48ca-b252-62ee220ba9dc None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:05.529591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 695/1380] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:05.535084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cec0-e34a-41a4-9345-3bd8ef35a63d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:05.535562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cec0-e34a-41a4-9345-3bd8ef35a63d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:05.535661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd2cec0-e34a-41a4-9345-3bd8ef35a63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:05.555039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-fcd2cec0-e34a-41a4-9345-3bd8ef35a63d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:05.556294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-fcd2cec0-e34a-41a4-9345-3bd8ef35a63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:05.564952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 686/1381] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:05.608197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196816b-9f85-48ca-b252-62ee220ba9dc None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:05.625418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196816b-9f85-48ca-b252-62ee220ba9dc None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:05.631273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7222545f-abd1-45a1-b610-d8e8f106a3ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:05.632054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222545f-abd1-45a1-b610-d8e8f106a3ae service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:05.632626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222545f-abd1-45a1-b610-d8e8f106a3ae service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:05.633000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7222545f-abd1-45a1-b610-d8e8f106a3ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:05.641835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222545f-abd1-45a1-b610-d8e8f106a3ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:05.643937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7222545f-abd1-45a1-b610-d8e8f106a3ae service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:05.656962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 696/1382] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:08.116122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfa63a-e5c1-440a-a5f6-b493dd402e0d None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:08.117011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfa63a-e5c1-440a-a5f6-b493dd402e0d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:08.117319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfa63a-e5c1-440a-a5f6-b493dd402e0d None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:08.118855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 687/1383] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:19:08.124915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72434c-8dff-4d95-b76b-1aa80de75a82 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:08.125424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72434c-8dff-4d95-b76b-1aa80de75a82 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:08.125527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72434c-8dff-4d95-b76b-1aa80de75a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:08.158515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-da72434c-8dff-4d95-b76b-1aa80de75a82 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:08.160450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-da72434c-8dff-4d95-b76b-1aa80de75a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:08.170088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 697/1384] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:08.218562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfa63a-e5c1-440a-a5f6-b493dd402e0d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:08.234221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfa63a-e5c1-440a-a5f6-b493dd402e0d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:08.240235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a4ac0-e2ba-4c8c-8e26-f0548ba1cf7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:08.240988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4ac0-e2ba-4c8c-8e26-f0548ba1cf7e service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:08.241510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4ac0-e2ba-4c8c-8e26-f0548ba1cf7e service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:08.241855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a4ac0-e2ba-4c8c-8e26-f0548ba1cf7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:08.251623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4ac0-e2ba-4c8c-8e26-f0548ba1cf7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:08.253565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a4ac0-e2ba-4c8c-8e26-f0548ba1cf7e service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:08.267799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 688/1385] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:10.692130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e0e0b-3e01-48ad-b464-983575fdaccd None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:10.692896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e0e0b-3e01-48ad-b464-983575fdaccd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:10.692896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e0e0b-3e01-48ad-b464-983575fdaccd None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:10.694681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 698/1386] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:10.700765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfaabc-b0dd-42a6-ad86-bcecb042448e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:10.701303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfaabc-b0dd-42a6-ad86-bcecb042448e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:10.701481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cfaabc-b0dd-42a6-ad86-bcecb042448e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:10.720973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-61cfaabc-b0dd-42a6-ad86-bcecb042448e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:10.722231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-61cfaabc-b0dd-42a6-ad86-bcecb042448e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:10.730728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 689/1387] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:10.779347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e0e0b-3e01-48ad-b464-983575fdaccd None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:10.797588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e0e0b-3e01-48ad-b464-983575fdaccd None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:10.804477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f757c8-52ab-4e38-b6f5-8d111ceb6706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:10.805619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f757c8-52ab-4e38-b6f5-8d111ceb6706 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:10.805974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f757c8-52ab-4e38-b6f5-8d111ceb6706 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:10.806330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f757c8-52ab-4e38-b6f5-8d111ceb6706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:10.817344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f757c8-52ab-4e38-b6f5-8d111ceb6706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:10.820046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f757c8-52ab-4e38-b6f5-8d111ceb6706 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:10.832704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 699/1388] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:13.201598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cabd5-872f-4cf3-8ff5-71296efa9f55 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:13.202152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cabd5-872f-4cf3-8ff5-71296efa9f55 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:13.202152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9cabd5-872f-4cf3-8ff5-71296efa9f55 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:13.203522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 690/1389] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:13.212305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f078e10-699d-4390-b93e-1af7fd58c1d9 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:13.212837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f078e10-699d-4390-b93e-1af7fd58c1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:13.212944 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f078e10-699d-4390-b93e-1af7fd58c1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:13.232248 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7f078e10-699d-4390-b93e-1af7fd58c1d9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:13.233479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7f078e10-699d-4390-b93e-1af7fd58c1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:13.242820 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 700/1390] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:13.288959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cabd5-872f-4cf3-8ff5-71296efa9f55 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:13.310587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9cabd5-872f-4cf3-8ff5-71296efa9f55 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:13.316360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e694e5-8bbf-45d8-a4af-39c7231d78d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:13.317005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e694e5-8bbf-45d8-a4af-39c7231d78d4 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:13.317286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e694e5-8bbf-45d8-a4af-39c7231d78d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:13.317564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e694e5-8bbf-45d8-a4af-39c7231d78d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:13.326990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e694e5-8bbf-45d8-a4af-39c7231d78d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:13.328604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e694e5-8bbf-45d8-a4af-39c7231d78d4 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:13.339176 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 691/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:15.732065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e124f-c000-4837-8a2f-c40a83fa2d1a None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:15.732671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e124f-c000-4837-8a2f-c40a83fa2d1a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:15.732671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e124f-c000-4837-8a2f-c40a83fa2d1a None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:15.734047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 701/1392] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:15.739524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf71d1-c022-4b86-84cb-c84d8bdd2b3b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:15.740066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf71d1-c022-4b86-84cb-c84d8bdd2b3b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:15.740174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cf71d1-c022-4b86-84cb-c84d8bdd2b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:15.759454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-56cf71d1-c022-4b86-84cb-c84d8bdd2b3b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:15.760714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-56cf71d1-c022-4b86-84cb-c84d8bdd2b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:15.768930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 692/1393] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:15.813851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e124f-c000-4837-8a2f-c40a83fa2d1a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:15.829151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e124f-c000-4837-8a2f-c40a83fa2d1a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:15.835189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108d529-c4a0-4554-ab79-c41487fe1088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:15.835862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d529-c4a0-4554-ab79-c41487fe1088 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:15.836150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d529-c4a0-4554-ab79-c41487fe1088 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:15.836428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108d529-c4a0-4554-ab79-c41487fe1088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:15.845037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108d529-c4a0-4554-ab79-c41487fe1088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:15.846631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108d529-c4a0-4554-ab79-c41487fe1088 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:15.856365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 702/1394] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:17.142501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9da889-d29e-4141-9fb9-9ba80450282b None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:17.143081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9da889-d29e-4141-9fb9-9ba80450282b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:17.143081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9da889-d29e-4141-9fb9-9ba80450282b None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:17.144511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 693/1395] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:17.150959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4fe9-66f9-471d-bbbe-60000f23d0c0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:17.151990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4fe9-66f9-471d-bbbe-60000f23d0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:17.152210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf4fe9-66f9-471d-bbbe-60000f23d0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:17.173379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-bcbf4fe9-66f9-471d-bbbe-60000f23d0c0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:17.174766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bcbf4fe9-66f9-471d-bbbe-60000f23d0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:17.183785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 703/1396] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:17.241242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf407d76-1748-4a67-87e6-b502109197cf None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:17.241601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf407d76-1748-4a67-87e6-b502109197cf None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:17.242080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf407d76-1748-4a67-87e6-b502109197cf None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:17.243618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 694/1397] 10.222.0.42 () {54 vars in 962 bytes} [Thu May 26 21:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:17.249557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913bbad-ced7-4252-85c2-e9f6c04b4e62 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:17.249557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913bbad-ced7-4252-85c2-e9f6c04b4e62 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:17.249819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913bbad-ced7-4252-85c2-e9f6c04b4e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:17.300587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8913bbad-ced7-4252-85c2-e9f6c04b4e62 None None] MFA Rules not processed for user `a98415ce308d41f6b8d5db6ca3f2ff19`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:17.301729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8913bbad-ced7-4252-85c2-e9f6c04b4e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:17.363042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 704/1398] 10.222.0.42 () {58 vars in 1059 bytes} [Thu May 26 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:17.370636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf407d76-1748-4a67-87e6-b502109197cf None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:17.375954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=87 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:19:17.382105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf407d76-1748-4a67-87e6-b502109197cf None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:17.385846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca934e3-9998-4a63-8ae2-1f658ddfbdb8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:17.386582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca934e3-9998-4a63-8ae2-1f658ddfbdb8 service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:17.386874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca934e3-9998-4a63-8ae2-1f658ddfbdb8 service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:17.387152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca934e3-9998-4a63-8ae2-1f658ddfbdb8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:17.396177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca934e3-9998-4a63-8ae2-1f658ddfbdb8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:17.398126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca934e3-9998-4a63-8ae2-1f658ddfbdb8 service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:17.408104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 695/1399] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:18.112035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51465f-1a69-4a5a-a369-5d5eddf6ff15 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:18.112331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51465f-1a69-4a5a-a369-5d5eddf6ff15 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:18.112633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51465f-1a69-4a5a-a369-5d5eddf6ff15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:18.163041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6c51465f-1a69-4a5a-a369-5d5eddf6ff15 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:18.164444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6c51465f-1a69-4a5a-a369-5d5eddf6ff15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:18.206924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:19:18.231231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 705/1400] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:18.275505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b32f-0a28-44d0-ac10-65eadd32aa66 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:18.276246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b32f-0a28-44d0-ac10-65eadd32aa66 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:18.276246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93b32f-0a28-44d0-ac10-65eadd32aa66 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:18.277706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 696/1401] 10.222.0.42 () {54 vars in 942 bytes} [Thu May 26 21:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:18.279912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c89c4a-c94e-400d-8146-aeff7d1a1b6a None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:18.280303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c89c4a-c94e-400d-8146-aeff7d1a1b6a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:18.280303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c89c4a-c94e-400d-8146-aeff7d1a1b6a None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:18.281644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 706/1402] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:18.284437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854010c1-ab23-48a9-b835-e0683d702d0b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:18.284824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854010c1-ab23-48a9-b835-e0683d702d0b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:18.285196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854010c1-ab23-48a9-b835-e0683d702d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:18.287155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e775fac-6adc-4c65-9d07-6b04828d51c1 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:18.287534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e775fac-6adc-4c65-9d07-6b04828d51c1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:18.287848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e775fac-6adc-4c65-9d07-6b04828d51c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:18.310301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0e775fac-6adc-4c65-9d07-6b04828d51c1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:18.311892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0e775fac-6adc-4c65-9d07-6b04828d51c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:18.320019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 707/1403] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:19:18.341978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-854010c1-ab23-48a9-b835-e0683d702d0b None None] MFA Rules not processed for user `ededf6d88dee4745978c8566940b2180`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:18.343223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-854010c1-ab23-48a9-b835-e0683d702d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:18.364215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e775fac-6adc-4c65-9d07-6b04828d51c1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:18.377481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e775fac-6adc-4c65-9d07-6b04828d51c1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:18.383156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:18.383835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:18.384114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:18.384517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:18.392996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:18.394606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:18.404824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 708/1404] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:19:18.421916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 697/1405] 10.222.0.42 () {58 vars in 1039 bytes} [Thu May 26 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:19:18.431910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:18.443359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ede2c35-db9e-43f8-beea-0bdb8121b125 service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:18.448609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb5011-245c-416b-b230-4204a4492bf2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:18.449252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5011-245c-416b-b230-4204a4492bf2 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:18.449600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5011-245c-416b-b230-4204a4492bf2 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:18.449840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5011-245c-416b-b230-4204a4492bf2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:18.458368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb5011-245c-416b-b230-4204a4492bf2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:18.460560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb5011-245c-416b-b230-4204a4492bf2 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:18.473705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 709/1406] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:19:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:19:20.806226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416b0c8-5e39-4eb2-b4fc-9ba887fa9a83 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:20.806776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416b0c8-5e39-4eb2-b4fc-9ba887fa9a83 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:20.806776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416b0c8-5e39-4eb2-b4fc-9ba887fa9a83 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:20.808472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 698/1407] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:20.814284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866957-a5c5-495d-a360-2a869f201fac None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:20.814799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866957-a5c5-495d-a360-2a869f201fac None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:20.814891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8866957-a5c5-495d-a360-2a869f201fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:20.833186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c8866957-a5c5-495d-a360-2a869f201fac None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:20.834951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c8866957-a5c5-495d-a360-2a869f201fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:20.843186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 710/1408] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:20.889226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416b0c8-5e39-4eb2-b4fc-9ba887fa9a83 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:20.911688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416b0c8-5e39-4eb2-b4fc-9ba887fa9a83 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:20.918021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13e0489-7494-4969-865d-a25c55a7278c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:20.918677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0489-7494-4969-865d-a25c55a7278c service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:20.918962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0489-7494-4969-865d-a25c55a7278c service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:20.919266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13e0489-7494-4969-865d-a25c55a7278c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:20.928177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e0489-7494-4969-865d-a25c55a7278c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:20.929748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13e0489-7494-4969-865d-a25c55a7278c service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:20.942049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 699/1409] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:30.974945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=27 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:19:31.372243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=55 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:19:42.705266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded34c7f-3702-4cd9-be07-b38591c696cb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:42.705902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded34c7f-3702-4cd9-be07-b38591c696cb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:42.705902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded34c7f-3702-4cd9-be07-b38591c696cb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:42.707197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 711/1410] 10.222.0.42 () {54 vars in 918 bytes} [Thu May 26 21:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:42.712897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d23aa-ae27-4435-bd2f-1d48f6db6863 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:42.713333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d23aa-ae27-4435-bd2f-1d48f6db6863 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:42.713447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6d23aa-ae27-4435-bd2f-1d48f6db6863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:42.731790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1e6d23aa-ae27-4435-bd2f-1d48f6db6863 None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:42.732979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1e6d23aa-ae27-4435-bd2f-1d48f6db6863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:42.741325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 700/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Thu May 26 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:42.755321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bb285-7ee1-4346-aa58-92fc33040f37 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:42.755608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bb285-7ee1-4346-aa58-92fc33040f37 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:42.756421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7bb285-7ee1-4346-aa58-92fc33040f37 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:42.757791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 712/1412] 10.222.0.42 () {54 vars in 918 bytes} [Thu May 26 21:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:42.762396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5114-c3b0-4503-b837-60149c756dfa None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:42.762683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5114-c3b0-4503-b837-60149c756dfa None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:42.762987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5114-c3b0-4503-b837-60149c756dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:42.781128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-77da5114-c3b0-4503-b837-60149c756dfa None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:42.782316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-77da5114-c3b0-4503-b837-60149c756dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:42.791265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 701/1413] 10.222.0.42 () {58 vars in 1015 bytes} [Thu May 26 21:19:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:45.428223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326ee8-bef0-4d57-a7bd-d0923ef8931a None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:45.428843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326ee8-bef0-4d57-a7bd-d0923ef8931a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:45.428843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac326ee8-bef0-4d57-a7bd-d0923ef8931a None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:45.430217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:45.435594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c845e-f9ed-4629-a353-9f8ad0abc39f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:45.436419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c845e-f9ed-4629-a353-9f8ad0abc39f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:45.436419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c845e-f9ed-4629-a353-9f8ad0abc39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:45.457244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ca9c845e-f9ed-4629-a353-9f8ad0abc39f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:45.458422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ca9c845e-f9ed-4629-a353-9f8ad0abc39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:45.466257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:45.508702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac326ee8-bef0-4d57-a7bd-d0923ef8931a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:45.523138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac326ee8-bef0-4d57-a7bd-d0923ef8931a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:45.528440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93bdf6-59d2-4b33-bfa8-9cd0b4257a7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:45.529116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93bdf6-59d2-4b33-bfa8-9cd0b4257a7f service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:45.529399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93bdf6-59d2-4b33-bfa8-9cd0b4257a7f service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:45.529674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93bdf6-59d2-4b33-bfa8-9cd0b4257a7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:45.537819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93bdf6-59d2-4b33-bfa8-9cd0b4257a7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:45.539397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93bdf6-59d2-4b33-bfa8-9cd0b4257a7f service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:45.549265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 714/1416] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:46.512080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b1ca2-0f94-4218-859e-7b360b5f215e None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:46.512661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b1ca2-0f94-4218-859e-7b360b5f215e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:46.512661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b1ca2-0f94-4218-859e-7b360b5f215e None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:46.513989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:46.518994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75ed0d-bcc0-4b57-9b4b-57a705d181aa None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:46.519443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75ed0d-bcc0-4b57-9b4b-57a705d181aa None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:46.519554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75ed0d-bcc0-4b57-9b4b-57a705d181aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:46.537974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-da75ed0d-bcc0-4b57-9b4b-57a705d181aa None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:46.539178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-da75ed0d-bcc0-4b57-9b4b-57a705d181aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:46.546897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:46.719477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b1ca2-0f94-4218-859e-7b360b5f215e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:46.734101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b1ca2-0f94-4218-859e-7b360b5f215e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:46.739480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04138c54-ff10-4623-bd6d-65b585a96268 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:46.740692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04138c54-ff10-4623-bd6d-65b585a96268 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:46.740974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04138c54-ff10-4623-bd6d-65b585a96268 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:46.741250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04138c54-ff10-4623-bd6d-65b585a96268 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:46.749324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04138c54-ff10-4623-bd6d-65b585a96268 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:46.750886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04138c54-ff10-4623-bd6d-65b585a96268 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:46.760566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 704/1419] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:47.684848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4e8c8-1c6c-4824-925e-40755c7130f3 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:47.685442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4e8c8-1c6c-4824-925e-40755c7130f3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:47.685442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4e8c8-1c6c-4824-925e-40755c7130f3 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:47.687013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:47.692031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632fcb2-1938-4c68-a757-85f9652d8088 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:47.692425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632fcb2-1938-4c68-a757-85f9652d8088 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:47.692588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632fcb2-1938-4c68-a757-85f9652d8088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:47.711191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3632fcb2-1938-4c68-a757-85f9652d8088 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:47.712417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3632fcb2-1938-4c68-a757-85f9652d8088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:47.720106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:47.868612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4e8c8-1c6c-4824-925e-40755c7130f3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:47.883601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4e8c8-1c6c-4824-925e-40755c7130f3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:47.888974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05179aa7-1afb-4206-af1c-1208b1c1d418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:47.889609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179aa7-1afb-4206-af1c-1208b1c1d418 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:47.889890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179aa7-1afb-4206-af1c-1208b1c1d418 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:47.890165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05179aa7-1afb-4206-af1c-1208b1c1d418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:47.898437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05179aa7-1afb-4206-af1c-1208b1c1d418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:47.900063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05179aa7-1afb-4206-af1c-1208b1c1d418 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:47.909481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 717/1422] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:48.821300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780d394-5181-4bff-9c88-d6c06f997648 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:48.821843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780d394-5181-4bff-9c88-d6c06f997648 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:48.821843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780d394-5181-4bff-9c88-d6c06f997648 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:48.823235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:48.828279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06832776-afcb-4ea8-931c-d5e93b3e8287 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:48.828700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06832776-afcb-4ea8-931c-d5e93b3e8287 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:48.828987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06832776-afcb-4ea8-931c-d5e93b3e8287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:48.847697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-06832776-afcb-4ea8-931c-d5e93b3e8287 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:48.848939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-06832776-afcb-4ea8-931c-d5e93b3e8287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:48.857602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:49.007172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780d394-5181-4bff-9c88-d6c06f997648 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:49.022026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780d394-5181-4bff-9c88-d6c06f997648 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:49.027353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5b17a-5866-4396-a5e6-f5e5e368dfa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:49.028027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5b17a-5866-4396-a5e6-f5e5e368dfa5 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:49.028309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5b17a-5866-4396-a5e6-f5e5e368dfa5 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:49.028583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5b17a-5866-4396-a5e6-f5e5e368dfa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:49.036565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5b17a-5866-4396-a5e6-f5e5e368dfa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:49.038152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5b17a-5866-4396-a5e6-f5e5e368dfa5 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:49.047662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 707/1425] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:49.954481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0728c-04f9-40f4-978b-3c001a4899b5 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:49.955014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0728c-04f9-40f4-978b-3c001a4899b5 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:49.955014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0728c-04f9-40f4-978b-3c001a4899b5 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:49.956427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:19:49.961957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc89ee-36fe-4e21-8524-01aa37f717ad None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:49.962352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc89ee-36fe-4e21-8524-01aa37f717ad None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:49.962513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbc89ee-36fe-4e21-8524-01aa37f717ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:49.980806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6cbc89ee-36fe-4e21-8524-01aa37f717ad None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:49.982029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6cbc89ee-36fe-4e21-8524-01aa37f717ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:49.989778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:19:50.138404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0728c-04f9-40f4-978b-3c001a4899b5 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:50.153132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0728c-04f9-40f4-978b-3c001a4899b5 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:50.158541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77049c9f-e212-47a0-944e-f76f0d1815d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:50.159173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77049c9f-e212-47a0-944e-f76f0d1815d1 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:50.159457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77049c9f-e212-47a0-944e-f76f0d1815d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:50.159732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77049c9f-e212-47a0-944e-f76f0d1815d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:50.168545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77049c9f-e212-47a0-944e-f76f0d1815d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:50.170143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77049c9f-e212-47a0-944e-f76f0d1815d1 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:50.179684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 720/1428] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:51.075070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c282e99-4bc7-41cd-b8dd-c1fa3d710aca None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:51.075576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c282e99-4bc7-41cd-b8dd-c1fa3d710aca None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:51.075673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c282e99-4bc7-41cd-b8dd-c1fa3d710aca None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:51.077030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:51.082484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ada00-8f0c-4741-9446-45745635fb9d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:51.082969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ada00-8f0c-4741-9446-45745635fb9d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:51.083077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ada00-8f0c-4741-9446-45745635fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:51.101611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-342ada00-8f0c-4741-9446-45745635fb9d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:51.102802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-342ada00-8f0c-4741-9446-45745635fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:51.110637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:51.259785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c282e99-4bc7-41cd-b8dd-c1fa3d710aca None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:51.275053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c282e99-4bc7-41cd-b8dd-c1fa3d710aca None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:51.280412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0c99b-d05b-4f7a-a665-e3ef3fb9f0f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:51.281051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0c99b-d05b-4f7a-a665-e3ef3fb9f0f3 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:51.281340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0c99b-d05b-4f7a-a665-e3ef3fb9f0f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:51.281622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0c99b-d05b-4f7a-a665-e3ef3fb9f0f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:51.290654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0c99b-d05b-4f7a-a665-e3ef3fb9f0f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:51.292198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0c99b-d05b-4f7a-a665-e3ef3fb9f0f3 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:51.301497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 710/1431] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:52.206363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6c9ce-ce64-4100-b466-56a9623edd7e None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:52.206819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6c9ce-ce64-4100-b466-56a9623edd7e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:52.206928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6c9ce-ce64-4100-b466-56a9623edd7e None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:52.208340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:52.213445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3f92-19b7-4d36-8487-ac22876ff0d6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:52.213859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3f92-19b7-4d36-8487-ac22876ff0d6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:52.214009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aa3f92-19b7-4d36-8487-ac22876ff0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:52.231439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-07aa3f92-19b7-4d36-8487-ac22876ff0d6 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:52.232721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-07aa3f92-19b7-4d36-8487-ac22876ff0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:52.240773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:52.389571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6c9ce-ce64-4100-b466-56a9623edd7e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:52.404135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6c9ce-ce64-4100-b466-56a9623edd7e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:52.409546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fbd538-9335-4222-ab22-5ade05587eee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:52.410176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbd538-9335-4222-ab22-5ade05587eee service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:52.410460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbd538-9335-4222-ab22-5ade05587eee service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:52.410738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fbd538-9335-4222-ab22-5ade05587eee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:52.418840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbd538-9335-4222-ab22-5ade05587eee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:52.420462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fbd538-9335-4222-ab22-5ade05587eee service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:52.430587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 723/1434] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:53.326416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51868-e56a-4862-9cb7-e70cd76465e3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:53.326938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51868-e56a-4862-9cb7-e70cd76465e3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:53.326938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba51868-e56a-4862-9cb7-e70cd76465e3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:53.328421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:53.333352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2a90e-4cc1-40bf-9563-2b014e137779 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:53.334415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2a90e-4cc1-40bf-9563-2b014e137779 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:53.334524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2a90e-4cc1-40bf-9563-2b014e137779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:53.352014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ede2a90e-4cc1-40bf-9563-2b014e137779 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:53.353210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ede2a90e-4cc1-40bf-9563-2b014e137779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:53.361465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:19:53.509803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba51868-e56a-4862-9cb7-e70cd76465e3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:53.524507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba51868-e56a-4862-9cb7-e70cd76465e3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:53.529919 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196efe1d-ccc4-4a43-9534-895fe043df2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:53.530562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196efe1d-ccc4-4a43-9534-895fe043df2f service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:53.530845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196efe1d-ccc4-4a43-9534-895fe043df2f service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:53.531122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196efe1d-ccc4-4a43-9534-895fe043df2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:53.539463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196efe1d-ccc4-4a43-9534-895fe043df2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:53.540966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196efe1d-ccc4-4a43-9534-895fe043df2f service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:53.550622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 713/1437] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:54.502936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c50cc-da7f-4d94-a1ad-590ccd03d901 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:54.503486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c50cc-da7f-4d94-a1ad-590ccd03d901 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:54.503486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c50cc-da7f-4d94-a1ad-590ccd03d901 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:54.504958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:54.510188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865e0f-a1ef-4369-8b30-1be37f13a145 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:54.510609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865e0f-a1ef-4369-8b30-1be37f13a145 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:54.510749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd865e0f-a1ef-4369-8b30-1be37f13a145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:54.530015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-cd865e0f-a1ef-4369-8b30-1be37f13a145 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:54.531271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-cd865e0f-a1ef-4369-8b30-1be37f13a145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:54.540121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:54.687963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c50cc-da7f-4d94-a1ad-590ccd03d901 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:54.703033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c50cc-da7f-4d94-a1ad-590ccd03d901 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:54.708470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe538-0885-431a-babb-4b5ed9dca4fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:54.709655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe538-0885-431a-babb-4b5ed9dca4fd service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:54.709937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe538-0885-431a-babb-4b5ed9dca4fd service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:54.710207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe538-0885-431a-babb-4b5ed9dca4fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:54.718540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe538-0885-431a-babb-4b5ed9dca4fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:54.720158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe538-0885-431a-babb-4b5ed9dca4fd service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:54.730027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 726/1440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:56.796961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025fac-dc35-443b-b2b9-42c6563cac24 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:56.797512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025fac-dc35-443b-b2b9-42c6563cac24 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:56.797512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9025fac-dc35-443b-b2b9-42c6563cac24 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:56.798882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:56.804662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6786f-efab-4859-84ce-3dbafa73462a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:56.805115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6786f-efab-4859-84ce-3dbafa73462a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:56.805212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd6786f-efab-4859-84ce-3dbafa73462a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:56.823670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1bd6786f-efab-4859-84ce-3dbafa73462a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:56.825029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1bd6786f-efab-4859-84ce-3dbafa73462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:56.832871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:56.870579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9025fac-dc35-443b-b2b9-42c6563cac24 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:56.882087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9025fac-dc35-443b-b2b9-42c6563cac24 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:56.885577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9402ff96-dbb9-4e3e-a15c-5e8546aa24be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:56.886421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ff96-dbb9-4e3e-a15c-5e8546aa24be service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:56.886710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ff96-dbb9-4e3e-a15c-5e8546aa24be service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:56.886989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9402ff96-dbb9-4e3e-a15c-5e8546aa24be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:56.897078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402ff96-dbb9-4e3e-a15c-5e8546aa24be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:56.898520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9402ff96-dbb9-4e3e-a15c-5e8546aa24be service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:56.908072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 716/1443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:19:59.325462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579f89b-9b06-42fd-9db2-e81a7c7ea828 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:59.326052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579f89b-9b06-42fd-9db2-e81a7c7ea828 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:59.326052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579f89b-9b06-42fd-9db2-e81a7c7ea828 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:59.327437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:19:59.332669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00f53c-3fc6-41d7-b300-d10329f11768 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:59.333097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00f53c-3fc6-41d7-b300-d10329f11768 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:59.333233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00f53c-3fc6-41d7-b300-d10329f11768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:59.351184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ce00f53c-3fc6-41d7-b300-d10329f11768 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:19:59.352415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ce00f53c-3fc6-41d7-b300-d10329f11768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:19:59.360201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:19:59.397592 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579f89b-9b06-42fd-9db2-e81a7c7ea828 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:19:59.408279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579f89b-9b06-42fd-9db2-e81a7c7ea828 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:19:59.412622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2280f8-6757-4fb2-b481-aea2e31f8704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:19:59.413299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2280f8-6757-4fb2-b481-aea2e31f8704 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:19:59.413585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2280f8-6757-4fb2-b481-aea2e31f8704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:19:59.413862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2280f8-6757-4fb2-b481-aea2e31f8704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:19:59.423370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2280f8-6757-4fb2-b481-aea2e31f8704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:19:59.424849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2280f8-6757-4fb2-b481-aea2e31f8704 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:19:59.435123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 729/1446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:01.988976 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bf855-a5be-492b-bbe3-83e4f8feb92d None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:01.989496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bf855-a5be-492b-bbe3-83e4f8feb92d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:01.989496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bf855-a5be-492b-bbe3-83e4f8feb92d None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:01.991349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:01.996794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c88bb-8091-4b1c-bcab-34c7cc0640fd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:01.997227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c88bb-8091-4b1c-bcab-34c7cc0640fd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:01.997324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c88bb-8091-4b1c-bcab-34c7cc0640fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:02.015610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0e8c88bb-8091-4b1c-bcab-34c7cc0640fd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:02.016874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0e8c88bb-8091-4b1c-bcab-34c7cc0640fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:02.025273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:02.063269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bf855-a5be-492b-bbe3-83e4f8feb92d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:02.073970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bf855-a5be-492b-bbe3-83e4f8feb92d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:02.077568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12c3cc-9900-4dc9-9d5a-f722026f8a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:02.078213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12c3cc-9900-4dc9-9d5a-f722026f8a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:02.078491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12c3cc-9900-4dc9-9d5a-f722026f8a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:02.078763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12c3cc-9900-4dc9-9d5a-f722026f8a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:02.088541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12c3cc-9900-4dc9-9d5a-f722026f8a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:02.089967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12c3cc-9900-4dc9-9d5a-f722026f8a4c service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:02.100178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 719/1449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:20:04.449755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5eb43-1d9d-4b45-a9b0-a1124f4835d1 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:04.450326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5eb43-1d9d-4b45-a9b0-a1124f4835d1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:04.450326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5eb43-1d9d-4b45-a9b0-a1124f4835d1 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:04.451678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:04.457226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58246c9-16d4-4fdf-ab08-bdf369703922 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:04.457720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58246c9-16d4-4fdf-ab08-bdf369703922 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:04.457820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58246c9-16d4-4fdf-ab08-bdf369703922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:04.476095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d58246c9-16d4-4fdf-ab08-bdf369703922 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:04.477328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d58246c9-16d4-4fdf-ab08-bdf369703922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:04.485295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:04.523846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5eb43-1d9d-4b45-a9b0-a1124f4835d1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:04.534622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5eb43-1d9d-4b45-a9b0-a1124f4835d1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:04.538717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c06358-bd37-43eb-a2a1-b280242ab91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:04.539353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c06358-bd37-43eb-a2a1-b280242ab91e service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:04.539640 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c06358-bd37-43eb-a2a1-b280242ab91e service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:04.539949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c06358-bd37-43eb-a2a1-b280242ab91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:04.549110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c06358-bd37-43eb-a2a1-b280242ab91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:04.550532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c06358-bd37-43eb-a2a1-b280242ab91e service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:04.561185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 732/1452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:06.832367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e864af-c890-4564-9fd3-8ca4cefec5a8 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:06.832957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e864af-c890-4564-9fd3-8ca4cefec5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:06.832957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e864af-c890-4564-9fd3-8ca4cefec5a8 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:06.834302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:06.839564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a67974-f763-4910-8a77-b97e66cd6b7a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:06.840061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a67974-f763-4910-8a77-b97e66cd6b7a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:06.840169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a67974-f763-4910-8a77-b97e66cd6b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:06.857916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-82a67974-f763-4910-8a77-b97e66cd6b7a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:06.859142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-82a67974-f763-4910-8a77-b97e66cd6b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:06.867484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:06.904090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e864af-c890-4564-9fd3-8ca4cefec5a8 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:06.915282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e864af-c890-4564-9fd3-8ca4cefec5a8 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:06.918839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3523a-4384-4d23-a28f-4f199f5dbb1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:06.919472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3523a-4384-4d23-a28f-4f199f5dbb1c service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:06.919776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3523a-4384-4d23-a28f-4f199f5dbb1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:06.920066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3523a-4384-4d23-a28f-4f199f5dbb1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:06.929439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3523a-4384-4d23-a28f-4f199f5dbb1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:06.930863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3523a-4384-4d23-a28f-4f199f5dbb1c service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:06.940794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 722/1455] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:09.252697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37b7e0-7882-4dfc-a7da-1a8bf13e99ec None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:09.253302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37b7e0-7882-4dfc-a7da-1a8bf13e99ec None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:09.253302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37b7e0-7882-4dfc-a7da-1a8bf13e99ec None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:09.254671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:09.262034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb05d-0058-4f19-9a9a-f1323c9afdb6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:09.262598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb05d-0058-4f19-9a9a-f1323c9afdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:09.263015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28eb05d-0058-4f19-9a9a-f1323c9afdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:09.283812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f28eb05d-0058-4f19-9a9a-f1323c9afdb6 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:09.285046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f28eb05d-0058-4f19-9a9a-f1323c9afdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:09.293679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:09.333879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37b7e0-7882-4dfc-a7da-1a8bf13e99ec None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:09.346108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37b7e0-7882-4dfc-a7da-1a8bf13e99ec None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:09.350463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fa5cb-5168-4ca8-865c-8ccc1865debc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:09.351150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa5cb-5168-4ca8-865c-8ccc1865debc service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:09.351440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa5cb-5168-4ca8-865c-8ccc1865debc service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:09.351759 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fa5cb-5168-4ca8-865c-8ccc1865debc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:09.362413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa5cb-5168-4ca8-865c-8ccc1865debc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:09.364029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fa5cb-5168-4ca8-865c-8ccc1865debc service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:09.376261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 735/1458] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:11.681984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78762c9a-2f7b-46e3-9c22-87447b9071ec None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:11.682570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78762c9a-2f7b-46e3-9c22-87447b9071ec None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:11.682570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78762c9a-2f7b-46e3-9c22-87447b9071ec None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:11.683979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:20:11.690252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2bf59-7a42-48ef-84e0-6b062d5bd2ec None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:11.690778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2bf59-7a42-48ef-84e0-6b062d5bd2ec None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:11.690882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2bf59-7a42-48ef-84e0-6b062d5bd2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:11.698120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=70 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:20:11.711396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-b4f2bf59-7a42-48ef-84e0-6b062d5bd2ec None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:11.712690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b4f2bf59-7a42-48ef-84e0-6b062d5bd2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:11.721804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:11.759725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78762c9a-2f7b-46e3-9c22-87447b9071ec None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:11.765232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=73 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:20:11.771280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78762c9a-2f7b-46e3-9c22-87447b9071ec None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:11.774953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d77ad6-b0e7-47e4-80f3-102a4d170085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:11.775605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77ad6-b0e7-47e4-80f3-102a4d170085 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:11.776457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77ad6-b0e7-47e4-80f3-102a4d170085 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:11.776457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77ad6-b0e7-47e4-80f3-102a4d170085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:11.787283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d77ad6-b0e7-47e4-80f3-102a4d170085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:11.789107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d77ad6-b0e7-47e4-80f3-102a4d170085 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:11.802718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 725/1461] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:14.093363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7c69a-6097-4612-9eb8-82ff69275ddb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:14.093976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7c69a-6097-4612-9eb8-82ff69275ddb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:14.093976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7c69a-6097-4612-9eb8-82ff69275ddb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:14.095268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:14.101921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911fc68-5952-4c5f-8844-5f7ecfabbe22 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:14.102711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911fc68-5952-4c5f-8844-5f7ecfabbe22 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:14.102888 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911fc68-5952-4c5f-8844-5f7ecfabbe22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:14.121655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d911fc68-5952-4c5f-8844-5f7ecfabbe22 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:14.122854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d911fc68-5952-4c5f-8844-5f7ecfabbe22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:14.131475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:14.167600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7c69a-6097-4612-9eb8-82ff69275ddb None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:14.184791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7c69a-6097-4612-9eb8-82ff69275ddb None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:14.189121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179aea2a-d929-4706-9f92-584a538c278f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:14.189783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aea2a-d929-4706-9f92-584a538c278f service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:14.190071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aea2a-d929-4706-9f92-584a538c278f service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:14.190350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179aea2a-d929-4706-9f92-584a538c278f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:14.200898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179aea2a-d929-4706-9f92-584a538c278f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:14.202357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179aea2a-d929-4706-9f92-584a538c278f service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:14.212948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 738/1464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:16.428874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8c615-11c3-4404-a937-f372146d4c1e None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:16.429442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8c615-11c3-4404-a937-f372146d4c1e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:16.429442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8c615-11c3-4404-a937-f372146d4c1e None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:16.430785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:16.436454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8426455-3827-4aa3-8f13-7d98f7dfeb7a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:16.436906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8426455-3827-4aa3-8f13-7d98f7dfeb7a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:16.437006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8426455-3827-4aa3-8f13-7d98f7dfeb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:16.455898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-b8426455-3827-4aa3-8f13-7d98f7dfeb7a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:16.457154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b8426455-3827-4aa3-8f13-7d98f7dfeb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:16.465279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:16.502891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8c615-11c3-4404-a937-f372146d4c1e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:16.514101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8c615-11c3-4404-a937-f372146d4c1e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:16.517836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8153e18-9eb6-4203-b1ee-9fb8c4fc1ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:16.518492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8153e18-9eb6-4203-b1ee-9fb8c4fc1ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:16.518806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8153e18-9eb6-4203-b1ee-9fb8c4fc1ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:16.519084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8153e18-9eb6-4203-b1ee-9fb8c4fc1ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:16.529328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8153e18-9eb6-4203-b1ee-9fb8c4fc1ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:16.530761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8153e18-9eb6-4203-b1ee-9fb8c4fc1ea0 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:16.541241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 728/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:18.765885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15d02-0a5c-4b18-9240-d9e973c16eff None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:18.766509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15d02-0a5c-4b18-9240-d9e973c16eff None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:18.766509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d15d02-0a5c-4b18-9240-d9e973c16eff None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:18.767960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:18.773478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1627a-6d3a-457a-b721-1905f8f938ec None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:18.773921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1627a-6d3a-457a-b721-1905f8f938ec None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:18.774035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df1627a-6d3a-457a-b721-1905f8f938ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:18.792816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5df1627a-6d3a-457a-b721-1905f8f938ec None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:18.794023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5df1627a-6d3a-457a-b721-1905f8f938ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:18.802560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:18.838329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15d02-0a5c-4b18-9240-d9e973c16eff None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:18.849775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d15d02-0a5c-4b18-9240-d9e973c16eff None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:18.853357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67be2f-ebad-4aa9-8d0c-8c42543c6d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:18.854001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67be2f-ebad-4aa9-8d0c-8c42543c6d00 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:18.854289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67be2f-ebad-4aa9-8d0c-8c42543c6d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:18.854568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67be2f-ebad-4aa9-8d0c-8c42543c6d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:18.863828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67be2f-ebad-4aa9-8d0c-8c42543c6d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:18.865921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67be2f-ebad-4aa9-8d0c-8c42543c6d00 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:18.875681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 741/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:28.787372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=17 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:20:28.871233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=17 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:20:52.797724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619be74-304d-4fee-8bd0-5746bbe67eb6 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:52.798330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619be74-304d-4fee-8bd0-5746bbe67eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:52.798330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619be74-304d-4fee-8bd0-5746bbe67eb6 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:52.799624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 730/1471] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:52.805330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff962f-71e8-4d20-92dd-ff4f2329c66f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:52.805860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff962f-71e8-4d20-92dd-ff4f2329c66f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:52.805941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff962f-71e8-4d20-92dd-ff4f2329c66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:52.824542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a6ff962f-71e8-4d20-92dd-ff4f2329c66f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:52.825757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a6ff962f-71e8-4d20-92dd-ff4f2329c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:52.834193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 742/1472] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:52.882380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b74b21-2590-4a06-b68e-8a95204beab0 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:52.882666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b74b21-2590-4a06-b68e-8a95204beab0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:52.882940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b74b21-2590-4a06-b68e-8a95204beab0 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:52.884347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 731/1473] 10.222.0.42 () {54 vars in 962 bytes} [Thu May 26 21:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:52.889778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9ed84-4132-489e-9304-eff74e881d07 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:52.890073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9ed84-4132-489e-9304-eff74e881d07 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:52.890353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9ed84-4132-489e-9304-eff74e881d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:52.909320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1ff9ed84-4132-489e-9304-eff74e881d07 None None] MFA Rules not processed for user `a98415ce308d41f6b8d5db6ca3f2ff19`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:52.911130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1ff9ed84-4132-489e-9304-eff74e881d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:52.917816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 743/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Thu May 26 21:20:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:52.925424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b74b21-2590-4a06-b68e-8a95204beab0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:52.937181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b74b21-2590-4a06-b68e-8a95204beab0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:52.940931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ea5e8-2a3f-4199-9223-825f0e90423b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:52.941678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ea5e8-2a3f-4199-9223-825f0e90423b service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:52.941959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ea5e8-2a3f-4199-9223-825f0e90423b service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:52.942261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ea5e8-2a3f-4199-9223-825f0e90423b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:52.952091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ea5e8-2a3f-4199-9223-825f0e90423b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:52.953529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ea5e8-2a3f-4199-9223-825f0e90423b service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:52.964716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 732/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:20:55.388544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e7e59-2111-426b-ab3f-9f841081b4eb None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:55.389137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e7e59-2111-426b-ab3f-9f841081b4eb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:55.389137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e7e59-2111-426b-ab3f-9f841081b4eb None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:55.390498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 744/1476] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:20:55.396806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103d2a6-897c-4d35-8ec6-011ee026bb6f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:55.397270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103d2a6-897c-4d35-8ec6-011ee026bb6f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:55.397397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6103d2a6-897c-4d35-8ec6-011ee026bb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:55.415841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6103d2a6-897c-4d35-8ec6-011ee026bb6f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:55.417060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6103d2a6-897c-4d35-8ec6-011ee026bb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:55.425365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 733/1477] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:20:55.449607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e7e59-2111-426b-ab3f-9f841081b4eb None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:55.463686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e7e59-2111-426b-ab3f-9f841081b4eb None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:55.468375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa3f0e8-96cb-4ed8-b83d-2d6081c90077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:55.469040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3f0e8-96cb-4ed8-b83d-2d6081c90077 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:55.469341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3f0e8-96cb-4ed8-b83d-2d6081c90077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:55.469600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa3f0e8-96cb-4ed8-b83d-2d6081c90077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:55.470354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3f0e8-96cb-4ed8-b83d-2d6081c90077 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:55.472228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-aaa3f0e8-96cb-4ed8-b83d-2d6081c90077 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:20:55.472228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:20:55.472228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:20:55.474019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa3f0e8-96cb-4ed8-b83d-2d6081c90077 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:55.495266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 745/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Thu May 26 21:20:55 2022] GET /identity/v3/projects => generated 2553 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:20:57.796402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634621f0-0064-4e1e-a60c-44bf9ce95bcc None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:57.797216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634621f0-0064-4e1e-a60c-44bf9ce95bcc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:57.797216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634621f0-0064-4e1e-a60c-44bf9ce95bcc None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:57.798353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 734/1479] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:20:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:20:57.805223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938bb2f-0c0c-4c25-85d2-8fa71b7ad932 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:57.805822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938bb2f-0c0c-4c25-85d2-8fa71b7ad932 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:57.805822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938bb2f-0c0c-4c25-85d2-8fa71b7ad932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:57.825097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f938bb2f-0c0c-4c25-85d2-8fa71b7ad932 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:20:57.826352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f938bb2f-0c0c-4c25-85d2-8fa71b7ad932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:20:57.834693 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 746/1480] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:20:57.882328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634621f0-0064-4e1e-a60c-44bf9ce95bcc None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:20:57.897699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634621f0-0064-4e1e-a60c-44bf9ce95bcc None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:20:57.903635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7c1b3-e7d5-4a89-a05f-e136137a9bc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:20:57.904343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7c1b3-e7d5-4a89-a05f-e136137a9bc5 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:20:57.904758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7c1b3-e7d5-4a89-a05f-e136137a9bc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:20:57.905029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7c1b3-e7d5-4a89-a05f-e136137a9bc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:20:57.914898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7c1b3-e7d5-4a89-a05f-e136137a9bc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:20:57.916610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7c1b3-e7d5-4a89-a05f-e136137a9bc5 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:20:57.928379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 735/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:21:00.443968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb562-de9d-41f4-a46b-02bb30e6147a None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:00.444628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb562-de9d-41f4-a46b-02bb30e6147a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:00.444786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610eb562-de9d-41f4-a46b-02bb30e6147a None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:00.446348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 747/1482] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:21:00.452623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65414a4a-b5d4-4ed9-bf7d-aa6cb3a08cdd None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:00.453241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65414a4a-b5d4-4ed9-bf7d-aa6cb3a08cdd None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:00.453455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65414a4a-b5d4-4ed9-bf7d-aa6cb3a08cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:00.474545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-65414a4a-b5d4-4ed9-bf7d-aa6cb3a08cdd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:21:00.475810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-65414a4a-b5d4-4ed9-bf7d-aa6cb3a08cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:21:00.484179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 736/1483] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:21:00.525187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb562-de9d-41f4-a46b-02bb30e6147a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:21:00.537244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610eb562-de9d-41f4-a46b-02bb30e6147a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:21:00.541240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c685f2-e7ff-434f-ac6c-10809bfe42c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:21:00.541906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c685f2-e7ff-434f-ac6c-10809bfe42c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:00.542189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c685f2-e7ff-434f-ac6c-10809bfe42c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:00.542466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c685f2-e7ff-434f-ac6c-10809bfe42c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:00.553398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c685f2-e7ff-434f-ac6c-10809bfe42c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:21:00.554862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c685f2-e7ff-434f-ac6c-10809bfe42c5 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:21:00.566030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 748/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:21:03.025330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4e5ec-badd-4e74-8b52-7a2010d9e428 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:03.025959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4e5ec-badd-4e74-8b52-7a2010d9e428 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:03.026306 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce4e5ec-badd-4e74-8b52-7a2010d9e428 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:03.028925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 737/1485] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:21:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:21:03.035726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856156-4254-42e1-a2cb-0314b40d62f4 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:03.036794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856156-4254-42e1-a2cb-0314b40d62f4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:03.037293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4856156-4254-42e1-a2cb-0314b40d62f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:03.069198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c4856156-4254-42e1-a2cb-0314b40d62f4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:21:03.071402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c4856156-4254-42e1-a2cb-0314b40d62f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:21:03.084519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 749/1486] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:21:03.126181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4e5ec-badd-4e74-8b52-7a2010d9e428 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:21:03.138451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce4e5ec-badd-4e74-8b52-7a2010d9e428 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:21:03.142393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d19c9a-967a-4e46-8bc1-8fabc05450c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:21:03.143066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19c9a-967a-4e46-8bc1-8fabc05450c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:03.143354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19c9a-967a-4e46-8bc1-8fabc05450c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:03.143633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d19c9a-967a-4e46-8bc1-8fabc05450c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:03.154030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19c9a-967a-4e46-8bc1-8fabc05450c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:21:03.155549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d19c9a-967a-4e46-8bc1-8fabc05450c8 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:21:03.167299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 738/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:21:05.778510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520f0bc-c462-41f0-bef1-5407a5e21359 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:05.779173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520f0bc-c462-41f0-bef1-5407a5e21359 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:05.779173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7520f0bc-c462-41f0-bef1-5407a5e21359 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:05.780991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 750/1488] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:21:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:21:05.787455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8867e7a-2809-4b27-bb19-3a6d813f75cd None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:05.788372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8867e7a-2809-4b27-bb19-3a6d813f75cd None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:05.788372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8867e7a-2809-4b27-bb19-3a6d813f75cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:05.807895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d8867e7a-2809-4b27-bb19-3a6d813f75cd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:21:05.809288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d8867e7a-2809-4b27-bb19-3a6d813f75cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:21:05.818577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 739/1489] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:21:05.860565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520f0bc-c462-41f0-bef1-5407a5e21359 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:21:05.873061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7520f0bc-c462-41f0-bef1-5407a5e21359 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:21:05.877011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa0beb9-f7dc-4606-b0a5-1b4f11d7b5f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:21:05.877688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0beb9-f7dc-4606-b0a5-1b4f11d7b5f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:05.877964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0beb9-f7dc-4606-b0a5-1b4f11d7b5f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:05.878234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa0beb9-f7dc-4606-b0a5-1b4f11d7b5f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:05.889682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0beb9-f7dc-4606-b0a5-1b4f11d7b5f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:21:05.891333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa0beb9-f7dc-4606-b0a5-1b4f11d7b5f4 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:21:05.902475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 751/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:21:08.736307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac003-eecc-41b3-8f56-a4913a55e2e1 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:08.736894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac003-eecc-41b3-8f56-a4913a55e2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:08.737219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eac003-eecc-41b3-8f56-a4913a55e2e1 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:08.739648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 740/1491] 10.222.0.42 () {54 vars in 926 bytes} [Thu May 26 21:21:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:21:08.745847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751c50-9802-4799-8cd5-385ed2a31903 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:08.746555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751c50-9802-4799-8cd5-385ed2a31903 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:08.746555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e751c50-9802-4799-8cd5-385ed2a31903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:08.766102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3e751c50-9802-4799-8cd5-385ed2a31903 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:21:08.767378 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3e751c50-9802-4799-8cd5-385ed2a31903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:21:08.776339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 752/1492] 10.222.0.42 () {58 vars in 1023 bytes} [Thu May 26 21:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:21:08.817573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eac003-eecc-41b3-8f56-a4913a55e2e1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:21:08.829038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eac003-eecc-41b3-8f56-a4913a55e2e1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:21:08.833030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73253-d151-48b5-8f8f-8f50d1394c55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:21:08.833708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73253-d151-48b5-8f8f-8f50d1394c55 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:21:08.833989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73253-d151-48b5-8f8f-8f50d1394c55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:21:08.834271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73253-d151-48b5-8f8f-8f50d1394c55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:21:08.844675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73253-d151-48b5-8f8f-8f50d1394c55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:21:08.846282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73253-d151-48b5-8f8f-8f50d1394c55 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:21:08.857540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 741/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:21:18.759874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=33 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:21:18.852505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=33 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:23:04.898137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ffdc9-8fd4-4b28-be69-9b4a9aa01e7c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:04.898765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ffdc9-8fd4-4b28-be69-9b4a9aa01e7c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:04.898765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ffdc9-8fd4-4b28-be69-9b4a9aa01e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:04.917121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a63ffdc9-8fd4-4b28-be69-9b4a9aa01e7c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:23:04.918343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a63ffdc9-8fd4-4b28-be69-9b4a9aa01e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:23:04.927247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 753/1494] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:23:04.942480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b696-57b1-4684-8e03-b8cde166146c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:04.943055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b696-57b1-4684-8e03-b8cde166146c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:04.943055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b696-57b1-4684-8e03-b8cde166146c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:04.960996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1e45b696-57b1-4684-8e03-b8cde166146c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:23:04.962113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1e45b696-57b1-4684-8e03-b8cde166146c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:23:04.969951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 742/1495] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:23:04.974798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ffdc9-8fd4-4b28-be69-9b4a9aa01e7c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:04.984507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63ffdc9-8fd4-4b28-be69-9b4a9aa01e7c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:04.989590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:04.990240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:04.990517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:04.990862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:04.991515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:23:04.991851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:04.992882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:04.992882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:23:04.992882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:23:04.993757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:05.005830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 754/1496] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:23:05.014042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b696-57b1-4684-8e03-b8cde166146c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:05.023460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b696-57b1-4684-8e03-b8cde166146c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:05.028059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:05.028715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.029144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.029428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.030135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:05.031929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING py.warnings [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:05.031929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: warnings.warn(msg) May 26 21:23:05.031929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: May 26 21:23:05.033123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:05.173799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 743/1497] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:23:05 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:23:05.178062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:05.188556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c5e8a-bd63-4e32-be12-bec75228982e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:05.248478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:05.249073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.249390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.249675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.250366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:05.251396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:05.251396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:23:05.251396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:23:05.252633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:05.430398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 755/1498] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:23:05 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:23:05.435039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:05.445627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba276e-21a1-4a62-bfd0-400ed6fcfd1f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:05.450398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:05.451044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.451327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.451601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.452350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:05.453492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING py.warnings [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:05.453492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: warnings.warn(msg) May 26 21:23:05.453492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: May 26 21:23:05.455020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:05.468351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 744/1499] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:23:05.473299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:05.483177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d233258-6636-4f60-89f9-0dced9b3bb8c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:05.487605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82854ddc-7f42-49ed-8a47-63a7df10c730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:05.488295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82854ddc-7f42-49ed-8a47-63a7df10c730 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.488655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82854ddc-7f42-49ed-8a47-63a7df10c730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.488858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82854ddc-7f42-49ed-8a47-63a7df10c730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.489575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82854ddc-7f42-49ed-8a47-63a7df10c730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:05.490589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-82854ddc-7f42-49ed-8a47-63a7df10c730 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:05.490589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:23:05.490589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:23:05.491460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82854ddc-7f42-49ed-8a47-63a7df10c730 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:05.504997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 756/1500] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:23:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:23:05.509066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:05.519445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc697216-6b38-4c31-b752-46af59d94ba0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:05.523995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb37b0-39ae-41e7-9c5c-daa185de9256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:05.524873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb37b0-39ae-41e7-9c5c-daa185de9256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.525156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb37b0-39ae-41e7-9c5c-daa185de9256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.525434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb37b0-39ae-41e7-9c5c-daa185de9256 admin admin] PATH_INFO: `/v3/projects/4252bf72d916409db49bb3c28087dc6b/users/062257abd3854c7fb8aa3738030f6e9d/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.541026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb37b0-39ae-41e7-9c5c-daa185de9256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4252bf72d916409db49bb3c28087dc6b, user_id=062257abd3854c7fb8aa3738030f6e9d, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:05.542058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING py.warnings [None req-0efb37b0-39ae-41e7-9c5c-daa185de9256 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:05.542058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: warnings.warn(msg) May 26 21:23:05.542058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: May 26 21:23:05.544573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb37b0-39ae-41e7-9c5c-daa185de9256 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:05.630141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 745/1501] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:23:05 2022] PUT /identity/v3/projects/4252bf72d916409db49bb3c28087dc6b/users/062257abd3854c7fb8aa3738030f6e9d/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:23:05.642185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e2c6-de84-4ad4-a7f0-e0ff09dcd36a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.642469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e2c6-de84-4ad4-a7f0-e0ff09dcd36a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.642746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350e2c6-de84-4ad4-a7f0-e0ff09dcd36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.670696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3350e2c6-de84-4ad4-a7f0-e0ff09dcd36a None None] MFA Rules not processed for user `062257abd3854c7fb8aa3738030f6e9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:23:05.671955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3350e2c6-de84-4ad4-a7f0-e0ff09dcd36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:23:05.738455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 757/1502] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:23:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:23:05.751689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7841f57-4d35-4c07-a5f4-2a154b6d2263 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.752009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7841f57-4d35-4c07-a5f4-2a154b6d2263 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.752290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7841f57-4d35-4c07-a5f4-2a154b6d2263 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.753702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 746/1503] 10.222.0.42 () {54 vars in 963 bytes} [Thu May 26 21:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:23:05.758604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94117488-768b-449c-b406-c90292bc03e1 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.758840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94117488-768b-449c-b406-c90292bc03e1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.759123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94117488-768b-449c-b406-c90292bc03e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.777435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-94117488-768b-449c-b406-c90292bc03e1 None None] MFA Rules not processed for user `b6cf996e1fdb4a458d06bfd23d082dec`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:23:05.778653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-94117488-768b-449c-b406-c90292bc03e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:23:05.823412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:23:05.831270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 758/1504] 10.222.0.42 () {58 vars in 1060 bytes} [Thu May 26 21:23:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:23:05.839421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7841f57-4d35-4c07-a5f4-2a154b6d2263 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:05.851086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7841f57-4d35-4c07-a5f4-2a154b6d2263 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:05.854817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:05.855458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.855760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.856076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.868425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:05.869864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:05.879729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 747/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:23:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:23:05.922218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c45fe-fec9-4894-a285-45989bb812c2 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.922620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c45fe-fec9-4894-a285-45989bb812c2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.922772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c45fe-fec9-4894-a285-45989bb812c2 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.924090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 759/1506] 10.222.0.42 () {52 vars in 829 bytes} [Thu May 26 21:23:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:23:05.927595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:05.991454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134eb3b-0073-4993-b4da-ef4639c3b865 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:05.996338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:05.997234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:05.997521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:05.997800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] PATH_INFO: `/v3/users/062257abd3854c7fb8aa3738030f6e9d` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:05.999117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=062257abd3854c7fb8aa3738030f6e9d)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:06.000214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING py.warnings [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:06.000214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: warnings.warn(msg) May 26 21:23:06.000214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: May 26 21:23:06.001940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:06.229462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062257abd3854c7fb8aa3738030f6e9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:06.248042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062257abd3854c7fb8aa3738030f6e9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:06.320877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062257abd3854c7fb8aa3738030f6e9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:06.321497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062257abd3854c7fb8aa3738030f6e9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:06.322196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062257abd3854c7fb8aa3738030f6e9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:06.331621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:23:06.341785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 748/1507] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:23:05 2022] DELETE /identity/v3/users/062257abd3854c7fb8aa3738030f6e9d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:23:06.346693 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:06.346978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:06.347253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:06.365600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:23:06.366814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:23:06.428748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 760/1508] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:23:06.439344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:06.440817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:23:06.507022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df382fa8-11f5-4549-8df1-4d9f4c35de07 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:06.510631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9aa774-36e1-46ed-94e3-0b9dbd3c3788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:06.511288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa774-36e1-46ed-94e3-0b9dbd3c3788 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:06.511568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa774-36e1-46ed-94e3-0b9dbd3c3788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:06.511872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9aa774-36e1-46ed-94e3-0b9dbd3c3788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:06.521316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa774-36e1-46ed-94e3-0b9dbd3c3788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:06.522854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9aa774-36e1-46ed-94e3-0b9dbd3c3788 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:06.533308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 749/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:23:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:23:07.078056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:23:07.079712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:23:07.095284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e5a1c-1c12-4c92-9d34-9552db613659 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:23:07.101422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:23:07.102360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:23:07.103579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:23:07.104027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] PATH_INFO: `/v3/projects/4252bf72d916409db49bb3c28087dc6b` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:23:07.105348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4252bf72d916409db49bb3c28087dc6b)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:23:07.106394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:23:07.106394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:23:07.106394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:23:07.107851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:23:07.310876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4252bf72d916409db49bb3c28087dc6b', 'request_id': 'req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:07.326478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4252bf72d916409db49bb3c28087dc6b', 'request_id': 'req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:07.333181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4252bf72d916409db49bb3c28087dc6b', 'request_id': 'req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:07.423859 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] The token cache is being invalidate because project 4252bf72d916409db49bb3c28087dc6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:23:07.424149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:07.424869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2afaebe8-ef22-4e61-98ae-fe80a54eb27b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:23:07.426414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 761/1510] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:23:07 2022] DELETE /identity/v3/projects/4252bf72d916409db49bb3c28087dc6b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:23:16.528700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=29 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:23:17.420024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:24:49.179572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e32a-d004-4571-848a-389925266c63 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:49.181824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e32a-d004-4571-848a-389925266c63 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:49.181824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e32a-d004-4571-848a-389925266c63 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:49.182021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:24:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:24:49.186744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6183ce-6653-493b-9c32-6406de115299 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:49.187171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6183ce-6653-493b-9c32-6406de115299 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:49.187295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6183ce-6653-493b-9c32-6406de115299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:49.205328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ed6183ce-6653-493b-9c32-6406de115299 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:24:49.207118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ed6183ce-6653-493b-9c32-6406de115299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:49.277596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:24:49.458294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600e32a-d004-4571-848a-389925266c63 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:49.460144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c600e32a-d004-4571-848a-389925266c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:49.541728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600e32a-d004-4571-848a-389925266c63 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:49.548211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b93d05-df8d-45a0-be38-c46d0975a575 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:49.548891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b93d05-df8d-45a0-be38-c46d0975a575 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:49.549264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b93d05-df8d-45a0-be38-c46d0975a575 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:49.549476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b93d05-df8d-45a0-be38-c46d0975a575 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:49.559379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b93d05-df8d-45a0-be38-c46d0975a575 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:49.561032 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b93d05-df8d-45a0-be38-c46d0975a575 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:49.572717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 751/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:24:50.464013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1588a9c-f273-4785-88de-bdab66e916f1 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:50.464595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1588a9c-f273-4785-88de-bdab66e916f1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:50.464707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1588a9c-f273-4785-88de-bdab66e916f1 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:50.466350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:24:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:24:50.472192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4942b-3302-44f9-b5e0-7ac8782aa916 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:50.472668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4942b-3302-44f9-b5e0-7ac8782aa916 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:50.472749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4942b-3302-44f9-b5e0-7ac8782aa916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:50.491200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-76c4942b-3302-44f9-b5e0-7ac8782aa916 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:24:50.492499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-76c4942b-3302-44f9-b5e0-7ac8782aa916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:50.500323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:24:50.655298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1588a9c-f273-4785-88de-bdab66e916f1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:50.671232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1588a9c-f273-4785-88de-bdab66e916f1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:50.678604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b852c1-6591-4f70-b3f9-861f670461de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:50.679277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b852c1-6591-4f70-b3f9-861f670461de service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:50.679572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b852c1-6591-4f70-b3f9-861f670461de service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:50.679891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b852c1-6591-4f70-b3f9-861f670461de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:50.692567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b852c1-6591-4f70-b3f9-861f670461de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:50.696024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b852c1-6591-4f70-b3f9-861f670461de service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:50.715243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 764/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:24:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:24:50.726377 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1299a-f294-4ce5-b1e4-aeec0f875a9b None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:50.726667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1299a-f294-4ce5-b1e4-aeec0f875a9b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:50.727052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1299a-f294-4ce5-b1e4-aeec0f875a9b None None] PATH_INFO: `/v3` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:50.728280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 753/1517] 10.222.0.60 () {54 vars in 925 bytes} [Thu May 26 21:24:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:24:50.734602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206b1ac-772a-495e-bd60-396b2f248250 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:50.734890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206b1ac-772a-495e-bd60-396b2f248250 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:50.735168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206b1ac-772a-495e-bd60-396b2f248250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:50.755042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0206b1ac-772a-495e-bd60-396b2f248250 None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:24:50.756334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0206b1ac-772a-495e-bd60-396b2f248250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:50.837705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 765/1518] 10.222.0.60 () {58 vars in 1015 bytes} [Thu May 26 21:24:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:24:55.018683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9540dc-db9f-4255-9ad0-4d8decece226 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:55.019357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9540dc-db9f-4255-9ad0-4d8decece226 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:55.019357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9540dc-db9f-4255-9ad0-4d8decece226 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:55.020721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 754/1519] 10.222.0.42 () {54 vars in 960 bytes} [Thu May 26 21:24:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:24:55.026394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6aeda7-ab3f-4651-979f-ece9f72a7ac2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:55.026955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6aeda7-ab3f-4651-979f-ece9f72a7ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:55.026955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6aeda7-ab3f-4651-979f-ece9f72a7ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:55.046370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-cc6aeda7-ab3f-4651-979f-ece9f72a7ac2 None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:24:55.047620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cc6aeda7-ab3f-4651-979f-ece9f72a7ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:24:55.057227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 766/1520] 10.222.0.42 () {58 vars in 1057 bytes} [Thu May 26 21:24:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:24:55.065104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9540dc-db9f-4255-9ad0-4d8decece226 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:24:55.081587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9540dc-db9f-4255-9ad0-4d8decece226 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:24:55.088495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30835da3-1dba-4b98-8fb2-2b183ba00463 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:24:55.088495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30835da3-1dba-4b98-8fb2-2b183ba00463 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:24:55.088495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30835da3-1dba-4b98-8fb2-2b183ba00463 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:24:55.088495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30835da3-1dba-4b98-8fb2-2b183ba00463 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:24:55.098253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30835da3-1dba-4b98-8fb2-2b183ba00463 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:24:55.099938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30835da3-1dba-4b98-8fb2-2b183ba00463 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:24:55.111188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 755/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:24:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:25:02.023262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b735f5-d23c-4961-a68e-ca8429a6d28d None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:02.023893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b735f5-d23c-4961-a68e-ca8429a6d28d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:02.023893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b735f5-d23c-4961-a68e-ca8429a6d28d None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:02.025202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:25:02.030441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2558f-3cd1-47bb-92e9-51c6446b404c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:02.031092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2558f-3cd1-47bb-92e9-51c6446b404c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:02.031092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a2558f-3cd1-47bb-92e9-51c6446b404c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:02.050453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-82a2558f-3cd1-47bb-92e9-51c6446b404c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:25:02.051681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-82a2558f-3cd1-47bb-92e9-51c6446b404c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:02.061290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:25:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:25:02.215164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b735f5-d23c-4961-a68e-ca8429a6d28d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:25:02.230445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b735f5-d23c-4961-a68e-ca8429a6d28d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:25:02.236268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc890aba-f03c-43a9-9a88-ab8f3be020b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:25:02.236508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc890aba-f03c-43a9-9a88-ab8f3be020b9 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:02.237348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc890aba-f03c-43a9-9a88-ab8f3be020b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:02.237623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc890aba-f03c-43a9-9a88-ab8f3be020b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:02.246498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc890aba-f03c-43a9-9a88-ab8f3be020b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:25:02.248150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc890aba-f03c-43a9-9a88-ab8f3be020b9 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:25:02.257539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 768/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:25:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:25:03.134244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de3f0-b8aa-4e39-be6d-9561921e4f71 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:03.134924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de3f0-b8aa-4e39-be6d-9561921e4f71 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:03.134924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792de3f0-b8aa-4e39-be6d-9561921e4f71 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:03.136537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 21:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:25:03.141908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c246ac7-63f1-4c7a-a850-9aa4dfa57d86 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:03.142499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c246ac7-63f1-4c7a-a850-9aa4dfa57d86 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:03.142499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c246ac7-63f1-4c7a-a850-9aa4dfa57d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:03.162253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-9c246ac7-63f1-4c7a-a850-9aa4dfa57d86 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:25:03.163494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-9c246ac7-63f1-4c7a-a850-9aa4dfa57d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:03.171857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 21:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:25:03.329681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792de3f0-b8aa-4e39-be6d-9561921e4f71 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:25:03.348471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792de3f0-b8aa-4e39-be6d-9561921e4f71 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:25:03.354859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7dec0-698f-408c-9716-462be19cb6bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:25:03.355787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dec0-698f-408c-9716-462be19cb6bc service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:03.356136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dec0-698f-408c-9716-462be19cb6bc service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:03.356460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7dec0-698f-408c-9716-462be19cb6bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:03.368100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7dec0-698f-408c-9716-462be19cb6bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:25:03.370133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7dec0-698f-408c-9716-462be19cb6bc service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:25:03.382769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 758/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:25:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:25:07.616588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271d09-5c51-4ceb-a314-e85e7e7ec671 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:07.617186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271d09-5c51-4ceb-a314-e85e7e7ec671 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:07.617410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271d09-5c51-4ceb-a314-e85e7e7ec671 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:07.619328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:25:07.625944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23152482-fc3f-46f0-9c72-0a24d723fa83 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:07.629005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23152482-fc3f-46f0-9c72-0a24d723fa83 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:07.629005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23152482-fc3f-46f0-9c72-0a24d723fa83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:07.646101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-23152482-fc3f-46f0-9c72-0a24d723fa83 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:25:07.647440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-23152482-fc3f-46f0-9c72-0a24d723fa83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:07.655823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:25:07.695317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271d09-5c51-4ceb-a314-e85e7e7ec671 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:25:07.697625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f8271d09-5c51-4ceb-a314-e85e7e7ec671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:07.759703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271d09-5c51-4ceb-a314-e85e7e7ec671 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:25:07.763971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223c7be-3d45-4e20-a61b-811dc05f04ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:25:07.764884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223c7be-3d45-4e20-a61b-811dc05f04ab service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:07.765200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223c7be-3d45-4e20-a61b-811dc05f04ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:07.765590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223c7be-3d45-4e20-a61b-811dc05f04ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:07.777897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223c7be-3d45-4e20-a61b-811dc05f04ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:25:07.779947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4223c7be-3d45-4e20-a61b-811dc05f04ab service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:25:07.786685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:25:07.793324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 771/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:25:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:25:10.105984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cab983-525b-426f-8d68-77e6eac01de2 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:10.106817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cab983-525b-426f-8d68-77e6eac01de2 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:10.106817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cab983-525b-426f-8d68-77e6eac01de2 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:10.107882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:25:10.113876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99283959-6fa4-4903-a49d-a4b5508683a2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:10.114491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99283959-6fa4-4903-a49d-a4b5508683a2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:10.114491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99283959-6fa4-4903-a49d-a4b5508683a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:10.133359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-99283959-6fa4-4903-a49d-a4b5508683a2 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:25:10.134617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-99283959-6fa4-4903-a49d-a4b5508683a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:10.143390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:25:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:25:10.181846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cab983-525b-426f-8d68-77e6eac01de2 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:25:10.183768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a5cab983-525b-426f-8d68-77e6eac01de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:10.197726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cab983-525b-426f-8d68-77e6eac01de2 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:25:10.201407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01164593-71c4-4041-9da7-489b92f31829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:25:10.202074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164593-71c4-4041-9da7-489b92f31829 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:10.202362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164593-71c4-4041-9da7-489b92f31829 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:10.202640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01164593-71c4-4041-9da7-489b92f31829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:10.213643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01164593-71c4-4041-9da7-489b92f31829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:25:10.215360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01164593-71c4-4041-9da7-489b92f31829 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:25:10.227134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 761/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:25:13.969219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbb499-ef1d-4437-8169-723503a14191 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:13.969733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbb499-ef1d-4437-8169-723503a14191 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:13.969733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbb499-ef1d-4437-8169-723503a14191 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:13.971814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:25:13.977066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dd505-95e7-41cc-9414-ffac22f8a387 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:13.977476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dd505-95e7-41cc-9414-ffac22f8a387 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:13.977696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29dd505-95e7-41cc-9414-ffac22f8a387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:13.995407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f29dd505-95e7-41cc-9414-ffac22f8a387 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:25:13.996660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f29dd505-95e7-41cc-9414-ffac22f8a387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:14.005299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:25:14.276399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbb499-ef1d-4437-8169-723503a14191 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:25:14.278068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-afbbb499-ef1d-4437-8169-723503a14191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:14.336794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbbb499-ef1d-4437-8169-723503a14191 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:25:14.340370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513e7e3-3125-41a4-934d-8d66dc0cecab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:25:14.341015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e7e3-3125-41a4-934d-8d66dc0cecab service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:14.341300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e7e3-3125-41a4-934d-8d66dc0cecab service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:14.341577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e7e3-3125-41a4-934d-8d66dc0cecab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:14.351014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513e7e3-3125-41a4-934d-8d66dc0cecab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:25:14.352476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513e7e3-3125-41a4-934d-8d66dc0cecab service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:25:14.362075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 774/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:25:16.576177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d3e8e-b103-4fa0-b0bf-929a5a9b63a3 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:16.576750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d3e8e-b103-4fa0-b0bf-929a5a9b63a3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:16.576750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d3e8e-b103-4fa0-b0bf-929a5a9b63a3 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:16.578139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 21:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:25:16.584071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41da10-08dc-414c-a652-ecaeb4400581 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:16.584529 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41da10-08dc-414c-a652-ecaeb4400581 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:16.584640 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41da10-08dc-414c-a652-ecaeb4400581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:16.602555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4f41da10-08dc-414c-a652-ecaeb4400581 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:25:16.603802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4f41da10-08dc-414c-a652-ecaeb4400581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:16.612311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 21:25:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:25:16.882927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d3e8e-b103-4fa0-b0bf-929a5a9b63a3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:25:16.884698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-de3d3e8e-b103-4fa0-b0bf-929a5a9b63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:25:16.892213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=65 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:25:16.898422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d3e8e-b103-4fa0-b0bf-929a5a9b63a3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:25:16.901974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ada21-8187-4cd6-b3bf-6ab2a307c70a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:25:16.902607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ada21-8187-4cd6-b3bf-6ab2a307c70a service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:25:16.914300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ada21-8187-4cd6-b3bf-6ab2a307c70a service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:25:16.914300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ada21-8187-4cd6-b3bf-6ab2a307c70a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:25:16.914300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ada21-8187-4cd6-b3bf-6ab2a307c70a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:25:16.915265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ada21-8187-4cd6-b3bf-6ab2a307c70a service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:25:16.926437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 764/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:25:26.597202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=32 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:25:26.940591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:49.641100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f8fe-479b-4492-ad22-fd4445fb4981 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.641753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f8fe-479b-4492-ad22-fd4445fb4981 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.641753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4f8fe-479b-4492-ad22-fd4445fb4981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.676992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8fd4f8fe-479b-4492-ad22-fd4445fb4981 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:49.678243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8fd4f8fe-479b-4492-ad22-fd4445fb4981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:49.692432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 776/1540] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:49.704180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fe582-dcdb-4acc-96c9-a8ca124d3891 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.705220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fe582-dcdb-4acc-96c9-a8ca124d3891 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.705739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96fe582-dcdb-4acc-96c9-a8ca124d3891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.729723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18336-f4ac-475f-a869-452ec6d29072 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.730079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18336-f4ac-475f-a869-452ec6d29072 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.730292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc18336-f4ac-475f-a869-452ec6d29072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.734537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d96fe582-dcdb-4acc-96c9-a8ca124d3891 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:49.736165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d96fe582-dcdb-4acc-96c9-a8ca124d3891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:49.756912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 765/1541] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:49.759770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ebc18336-f4ac-475f-a869-452ec6d29072 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:49.759770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ebc18336-f4ac-475f-a869-452ec6d29072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:49.768659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fe582-dcdb-4acc-96c9-a8ca124d3891 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:49.774351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 777/1542] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:49.779545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96fe582-dcdb-4acc-96c9-a8ca124d3891 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:49.784691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af769-acfb-4db8-b2f9-1cf6f9ddc25f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.785022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af769-acfb-4db8-b2f9-1cf6f9ddc25f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.785305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af769-acfb-4db8-b2f9-1cf6f9ddc25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.795247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:49.796822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.797344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.797842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.798836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:49.799347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:49.801192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING py.warnings [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 21:26:49.801192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: warnings.warn(msg) May 26 21:26:49.801192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: May 26 21:26:49.803653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39061625-5ca0-41c7-b2fc-193916813c71 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:49.821878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-9c2af769-acfb-4db8-b2f9-1cf6f9ddc25f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:49.823127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-9c2af769-acfb-4db8-b2f9-1cf6f9ddc25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:49.824903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 766/1543] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:49.826656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0009517-1b38-45e3-97ac-6da1d8a3c7c1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.827198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0009517-1b38-45e3-97ac-6da1d8a3c7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.827707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0009517-1b38-45e3-97ac-6da1d8a3c7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.842374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 778/1544] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:26:49.843305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af769-acfb-4db8-b2f9-1cf6f9ddc25f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:49.854784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b0009517-1b38-45e3-97ac-6da1d8a3c7c1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:49.857020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b0009517-1b38-45e3-97ac-6da1d8a3c7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:49.863771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af769-acfb-4db8-b2f9-1cf6f9ddc25f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:49.871449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 767/1545] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:49.872678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0009517-1b38-45e3-97ac-6da1d8a3c7c1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:49.881368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec338a-d759-48bc-b551-4a81897bda5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:49.882064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec338a-d759-48bc-b551-4a81897bda5e admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.882351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec338a-d759-48bc-b551-4a81897bda5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.882629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec338a-d759-48bc-b551-4a81897bda5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.883335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec338a-d759-48bc-b551-4a81897bda5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:49.884542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-faec338a-d759-48bc-b551-4a81897bda5e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:26:49.884542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:26:49.884542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:26:49.885627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec338a-d759-48bc-b551-4a81897bda5e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:49.889096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0009517-1b38-45e3-97ac-6da1d8a3c7c1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:49.900771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996635d-39cb-41e9-97cf-f9f7d140d032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:49.901799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996635d-39cb-41e9-97cf-f9f7d140d032 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.902294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996635d-39cb-41e9-97cf-f9f7d140d032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.903330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996635d-39cb-41e9-97cf-f9f7d140d032 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.904324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996635d-39cb-41e9-97cf-f9f7d140d032 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:49.904850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996635d-39cb-41e9-97cf-f9f7d140d032 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:49.907555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996635d-39cb-41e9-97cf-f9f7d140d032 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:49.936841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 768/1546] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:49.939062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba1feb-4b6d-4fd6-b4bb-1b04537f2fe9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.939562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba1feb-4b6d-4fd6-b4bb-1b04537f2fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.940062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba1feb-4b6d-4fd6-b4bb-1b04537f2fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:49.963107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-73ba1feb-4b6d-4fd6-b4bb-1b04537f2fe9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:49.964872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-73ba1feb-4b6d-4fd6-b4bb-1b04537f2fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:49.980190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 769/1547] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:49.989961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8390d5e-3ab2-4cf6-ab1e-0570ac6b04d7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:49.989961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8390d5e-3ab2-4cf6-ab1e-0570ac6b04d7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:49.989961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8390d5e-3ab2-4cf6-ab1e-0570ac6b04d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.016547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-e8390d5e-3ab2-4cf6-ab1e-0570ac6b04d7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.018999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e8390d5e-3ab2-4cf6-ab1e-0570ac6b04d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.028246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 770/1548] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.029572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8390d5e-3ab2-4cf6-ab1e-0570ac6b04d7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.045895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8390d5e-3ab2-4cf6-ab1e-0570ac6b04d7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.055118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a269c9-9929-4780-a98e-8003f6f79e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.056498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a269c9-9929-4780-a98e-8003f6f79e96 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.057064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a269c9-9929-4780-a98e-8003f6f79e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.057601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a269c9-9929-4780-a98e-8003f6f79e96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.058609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a269c9-9929-4780-a98e-8003f6f79e96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.061095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a269c9-9929-4780-a98e-8003f6f79e96 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.065635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 779/1549] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:49 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:50.067155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36525e-e8f0-4f8a-833b-922a250f6665 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.067450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36525e-e8f0-4f8a-833b-922a250f6665 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.067733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36525e-e8f0-4f8a-833b-922a250f6665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.094311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0a36525e-e8f0-4f8a-833b-922a250f6665 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.095538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0a36525e-e8f0-4f8a-833b-922a250f6665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.182153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 771/1550] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:50.183049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 780/1551] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.184042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36525e-e8f0-4f8a-833b-922a250f6665 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.184217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386fb3e-f110-4075-b8c9-40d9c460677c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.184404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386fb3e-f110-4075-b8c9-40d9c460677c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.184676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5386fb3e-f110-4075-b8c9-40d9c460677c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.195222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36525e-e8f0-4f8a-833b-922a250f6665 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.208489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5386fb3e-f110-4075-b8c9-40d9c460677c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.210617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5386fb3e-f110-4075-b8c9-40d9c460677c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.258084 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.258755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.259035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.259310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.260075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:50.260387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.262006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.274533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 781/1552] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:50.276283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.287208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4971c9c0-6e89-4dda-a320-dd7ac8dfddff admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.288221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 772/1553] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.289600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053ee32-8f59-48d1-a5c4-0fa8857e22fe None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.289889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053ee32-8f59-48d1-a5c4-0fa8857e22fe None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.290164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053ee32-8f59-48d1-a5c4-0fa8857e22fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.293168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd28f9-3947-40f3-a5b0-906cef0ae9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.293892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd28f9-3947-40f3-a5b0-906cef0ae9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.294278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd28f9-3947-40f3-a5b0-906cef0ae9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.294565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd28f9-3947-40f3-a5b0-906cef0ae9d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.295270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd28f9-3947-40f3-a5b0-906cef0ae9d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.298585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd28f9-3947-40f3-a5b0-906cef0ae9d4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.309215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f053ee32-8f59-48d1-a5c4-0fa8857e22fe None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.310442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f053ee32-8f59-48d1-a5c4-0fa8857e22fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.319027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 773/1554] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.320380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521644-3d25-40b9-8264-d23d47de2312 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.320671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521644-3d25-40b9-8264-d23d47de2312 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.320958 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521644-3d25-40b9-8264-d23d47de2312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.342795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ed521644-3d25-40b9-8264-d23d47de2312 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.344071 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ed521644-3d25-40b9-8264-d23d47de2312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.352618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 774/1555] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.353885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3ea3d-510f-4e6b-99d2-ae777b77ab86 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.354207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3ea3d-510f-4e6b-99d2-ae777b77ab86 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.354487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d3ea3d-510f-4e6b-99d2-ae777b77ab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.373357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-75d3ea3d-510f-4e6b-99d2-ae777b77ab86 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.374545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-75d3ea3d-510f-4e6b-99d2-ae777b77ab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.382876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 775/1556] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.383868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3ea3d-510f-4e6b-99d2-ae777b77ab86 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.393937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d3ea3d-510f-4e6b-99d2-ae777b77ab86 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.396721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 782/1557] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:50.397401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d21a5-8546-427e-ae86-ef39b5460266 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.397834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d21a5-8546-427e-ae86-ef39b5460266 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.398196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d21a5-8546-427e-ae86-ef39b5460266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.401905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.408444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.408444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.408444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.408444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.408444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING py.warnings [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:26:50.408444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: warnings.warn(msg) May 26 21:26:50.408444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: May 26 21:26:50.409445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.420777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-779d21a5-8546-427e-ae86-ef39b5460266 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.421994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-779d21a5-8546-427e-ae86-ef39b5460266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.430436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 783/1558] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.431350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d21a5-8546-427e-ae86-ef39b5460266 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.441306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d21a5-8546-427e-ae86-ef39b5460266 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.446603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4e0c3-3ae9-48ad-8ac4-348237a421e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.446772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4e0c3-3ae9-48ad-8ac4-348237a421e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.446901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4e0c3-3ae9-48ad-8ac4-348237a421e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.447169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4e0c3-3ae9-48ad-8ac4-348237a421e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.447886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4e0c3-3ae9-48ad-8ac4-348237a421e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.449792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4e0c3-3ae9-48ad-8ac4-348237a421e3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.488052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 776/1559] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:50.488905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.498853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9de311-a30f-4d45-b1d2-0e150e3b8144 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.503769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53badcd3-d7fa-4a95-9c8e-9d4acc64318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.504421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53badcd3-d7fa-4a95-9c8e-9d4acc64318f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.504701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53badcd3-d7fa-4a95-9c8e-9d4acc64318f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.504972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53badcd3-d7fa-4a95-9c8e-9d4acc64318f admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.505722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53badcd3-d7fa-4a95-9c8e-9d4acc64318f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:50.506029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53badcd3-d7fa-4a95-9c8e-9d4acc64318f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.507616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53badcd3-d7fa-4a95-9c8e-9d4acc64318f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.520474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 777/1560] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:50.521862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b85c2-0086-49c0-8dd4-d52323189a2d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.522148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b85c2-0086-49c0-8dd4-d52323189a2d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.522426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b85c2-0086-49c0-8dd4-d52323189a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.540503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d96b85c2-0086-49c0-8dd4-d52323189a2d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.541693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d96b85c2-0086-49c0-8dd4-d52323189a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.545791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 784/1561] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/projects => generated 363 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:50.547113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11857-c551-41d7-85ef-ef80ff0e4a74 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.547399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11857-c551-41d7-85ef-ef80ff0e4a74 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.547676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11857-c551-41d7-85ef-ef80ff0e4a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.552750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 778/1562] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.557213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b00a4d-a2f8-4166-a5fb-fed52298335a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.557456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b00a4d-a2f8-4166-a5fb-fed52298335a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.557728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b00a4d-a2f8-4166-a5fb-fed52298335a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.561225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:50.566772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-0ba11857-c551-41d7-85ef-ef80ff0e4a74 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.568100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0ba11857-c551-41d7-85ef-ef80ff0e4a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.576534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-45b00a4d-a2f8-4166-a5fb-fed52298335a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:50.577875 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-45b00a4d-a2f8-4166-a5fb-fed52298335a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:50.604092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:50.630950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 785/1563] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.632876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba11857-c551-41d7-85ef-ef80ff0e4a74 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.642340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 779/1564] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:50.642660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba11857-c551-41d7-85ef-ef80ff0e4a74 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.643190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b00a4d-a2f8-4166-a5fb-fed52298335a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.649530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.650195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.650476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.650749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.651478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.653119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.655594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b00a4d-a2f8-4166-a5fb-fed52298335a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.660410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.661074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.661358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.661632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.662357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:50.662667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.663684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 786/1565] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:50.664292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.664593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.675799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d44648-0d6a-4205-832d-b04bab3eb503 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.680633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 780/1566] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:50.680633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.684632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.685314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.685629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.686124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.686807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af2c262-70d8-4b20-935e-9e19ed9a1a00 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.686926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.688681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.693358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.694017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.694301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.694575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.695271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.697238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.700668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 787/1567] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:50.701810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.713250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94984d-4aae-40b8-a990-f049ed4d30ff admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.717807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.718470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.718748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.719023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.719808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.721726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.842008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 781/1568] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:50.842846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.852943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 788/1569] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/users => generated 282 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:50.854280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.868047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263aa1c-4224-4717-9418-06635cc3c42d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.914649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b968ff4-d0c1-41c5-a740-0ebc41623720 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.921307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.921307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.921307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.921307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.921738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:50.921827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.923423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.934853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.935859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 782/1570] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:50.936051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.936539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.936753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.937279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.938580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:50.940011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.943713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.954177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe28ff-2f15-4ffe-9829-64c91a0adef9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.954732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.955412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.956258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.956258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.958297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:26:50.958630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:50.960285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:50.967200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 789/1571] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:50.969107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.977335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 783/1572] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:26:50.984610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:50.986951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28f4fb-df2a-4aa9-911b-329343bee33f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.994482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b9a6d-c813-4b6e-b5fa-d7b21672fa9d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:50.995996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:50.997459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:50.998029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:50.998604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:50.999717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.000219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.000519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.000790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.001063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.001758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.003432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.004379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.018361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 790/1573] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:51.019277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.030326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201a00b-33fa-404e-89c6-686958a06291 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.036004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.036004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.036415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.036694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.037430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.039069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.051085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 791/1574] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:51.052316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.063560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d232495-e0f8-41f8-90b6-114864ab1806 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.069407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.070098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.070387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.070665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.071397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.073362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.094955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 784/1575] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:50 2022] POST /identity/v3/projects => generated 393 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:51.096076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.160016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 792/1576] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/users => generated 281 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:51.160016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.170669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51816a-95f0-4f61-9f9a-00e604352567 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.171718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:51.184383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea332f9-502b-4ea3-912c-0d9ed884cee2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.190454 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.191770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.192057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.192333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.193078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.194808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.199795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:51.205617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 785/1577] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:51.206474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.216955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f40944-9518-4d04-b046-2f8fc5df5514 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.224272 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.224957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.225240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.225517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.226214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.228178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.255671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.256370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.256654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.256931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.257658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.259586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.363399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 786/1578] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:51.364285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.382678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 793/1579] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/projects => generated 373 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:51.384730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.440183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407dfc0-d571-4411-816d-dea559ae3ce0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.463020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b6b78-0ccc-4421-b582-4c8e9b71f701 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.469409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.470468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.470749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.471021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] PATH_INFO: `/v3/projects/199159bff7e5433ca3da168465c2e22c/users/71302686cd59424ba952b49c0fad3577/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.488635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199159bff7e5433ca3da168465c2e22c, user_id=71302686cd59424ba952b49c0fad3577, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.489703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:26:51.489703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:26:51.489703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:26:51.491653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.519726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.520445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.520727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.521004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.521711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.523625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.564082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 794/1580] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:51 2022] PUT /identity/v3/projects/199159bff7e5433ca3da168465c2e22c/users/71302686cd59424ba952b49c0fad3577/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:51.565427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.639323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98c610c-62dc-45dc-b4e4-86f00ae139b5 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.644386 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.645297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.645583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.645868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] PATH_INFO: `/v3/projects/a3c67e60a5274e179ffd719af7cc9457/users/5c66120fd1124221b4a5552ddb6cd145/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.653273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 787/1581] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/projects => generated 385 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:51.654281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.663424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c67e60a5274e179ffd719af7cc9457, user_id=5c66120fd1124221b4a5552ddb6cd145, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.666145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.741598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a7795a-c883-447e-9f28-6d4b22728776 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.745715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:51.749277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 795/1582] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:51 2022] PUT /identity/v3/projects/a3c67e60a5274e179ffd719af7cc9457/users/5c66120fd1124221b4a5552ddb6cd145/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:51.760172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.773218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:51.808561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.810126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.810632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.810901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.811644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.813828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.824983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d8324-ed0d-4240-a094-146a0cc18c3e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.829907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.830603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.830903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.831176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.832493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.834143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.845030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 796/1583] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:51.845926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.862469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51e5ee-1f2e-4b7a-ae50-ce43e3afa5be admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.868960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.870722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.871098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.871428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.872434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.874403 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.885538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 797/1584] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:51.889165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.901055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8697f29-d97c-4c37-9cc9-4621fb378044 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.911395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 788/1585] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:51.912131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04931622-d3db-435e-a5d8-ce1765f56f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.912131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04931622-d3db-435e-a5d8-ce1765f56f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.912131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04931622-d3db-435e-a5d8-ce1765f56f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.912131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04931622-d3db-435e-a5d8-ce1765f56f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.912476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:51.912574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04931622-d3db-435e-a5d8-ce1765f56f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.912574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04931622-d3db-435e-a5d8-ce1765f56f6d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.925988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8567f4-5f0d-4e97-bfed-1e06d90063fb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:51.933023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94711ad-faca-45f4-857a-ad8aa091d5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:51.933815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94711ad-faca-45f4-857a-ad8aa091d5bc admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.934512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94711ad-faca-45f4-857a-ad8aa091d5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.934512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94711ad-faca-45f4-857a-ad8aa091d5bc admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:51.935285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94711ad-faca-45f4-857a-ad8aa091d5bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:51.938293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94711ad-faca-45f4-857a-ad8aa091d5bc admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:51.992591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 798/1586] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:51.994173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e593-9ccf-4a58-9174-b0d8428c11c6 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:51.994599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e593-9ccf-4a58-9174-b0d8428c11c6 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:51.994881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2573e593-9ccf-4a58-9174-b0d8428c11c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.015590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-2573e593-9ccf-4a58-9174-b0d8428c11c6 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:52.016869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2573e593-9ccf-4a58-9174-b0d8428c11c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:52.025641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 799/1587] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:52.026536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573e593-9ccf-4a58-9174-b0d8428c11c6 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.041903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2573e593-9ccf-4a58-9174-b0d8428c11c6 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.048410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.049238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.049733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.050045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.050844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.052921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.083737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 789/1588] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:51 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:52.085468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6de49-aa5c-4961-9b2f-7d9f4e5afc32 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.085761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6de49-aa5c-4961-9b2f-7d9f4e5afc32 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.086039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6de49-aa5c-4961-9b2f-7d9f4e5afc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.121421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5dd6de49-aa5c-4961-9b2f-7d9f4e5afc32 None None] MFA Rules not processed for user `5c66120fd1124221b4a5552ddb6cd145`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:52.123228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5dd6de49-aa5c-4961-9b2f-7d9f4e5afc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:52.168989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 800/1589] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:52 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:52.170144 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.181757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa00171-0dc8-45d3-a319-3404ffe9cf29 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.186736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.187411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.187696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.188000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.188950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.190583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.202971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 801/1590] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.203231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.209419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 790/1591] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:52.210930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6de49-aa5c-4961-9b2f-7d9f4e5afc32 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.221405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e1869c-4971-4a8e-b430-05dde08f241e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.226927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.227241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6de49-aa5c-4961-9b2f-7d9f4e5afc32 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.227894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.228180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.228459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] PATH_INFO: `/v3/projects/d2d73f3700b549bdbcc6296883cbb5a0/users/38f9c3f296de43cea60525957baed325/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.234146 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.235024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.235435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.236769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.237415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.240007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.249372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2d73f3700b549bdbcc6296883cbb5a0, user_id=38f9c3f296de43cea60525957baed325, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.252666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.260430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 791/1592] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.261654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.275337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2b075-bdca-428d-9883-f4b85bde1d32 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.280665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.281361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.281648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.281929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.282667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.284400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.300561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 792/1593] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.300561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.357683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 802/1594] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:52 2022] PUT /identity/v3/projects/d2d73f3700b549bdbcc6296883cbb5a0/users/38f9c3f296de43cea60525957baed325/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:52.359322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.365127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:52.371495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a357-3c0d-4c57-b183-e1c309fee810 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.433407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae2eda-30c5-4eb3-8f00-ba162670b8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.434110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2eda-30c5-4eb3-8f00-ba162670b8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.434356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2eda-30c5-4eb3-8f00-ba162670b8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.434628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2eda-30c5-4eb3-8f00-ba162670b8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.454202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64027d-7fab-431f-80e1-ac42508f9548 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.460616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.461465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.461877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.462343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.463696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.467275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.487275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 803/1595] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.489177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.501934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc59e23-b8bc-4f32-a8e4-2ae1587026bf admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.508828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.509626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.509965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.510327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.511303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.513296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.526118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 804/1596] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.528035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.540437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63268-ed16-4395-885f-a79ab375ca86 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.545759 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.546676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.546961 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.547236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] PATH_INFO: `/v3/projects/30a7e028890f440780a841bac629a2eb/users/164ce06a5a1a4cf7bab104b1270c9de1/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.550319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:52.562248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae2eda-30c5-4eb3-8f00-ba162670b8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.565078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae2eda-30c5-4eb3-8f00-ba162670b8a0 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.567389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a7e028890f440780a841bac629a2eb, user_id=164ce06a5a1a4cf7bab104b1270c9de1, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.570419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.583827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 793/1597] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:52.585772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecf961-baf2-4f30-ae41-fa7a0cd3220d None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.586274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecf961-baf2-4f30-ae41-fa7a0cd3220d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.586778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecf961-baf2-4f30-ae41-fa7a0cd3220d None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.589475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 794/1598] 10.222.0.42 () {54 vars in 961 bytes} [Thu May 26 21:26:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:26:52.590878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecf961-baf2-4f30-ae41-fa7a0cd3220d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.612022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecf961-baf2-4f30-ae41-fa7a0cd3220d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.615950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 805/1599] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:52 2022] PUT /identity/v3/projects/30a7e028890f440780a841bac629a2eb/users/164ce06a5a1a4cf7bab104b1270c9de1/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:52.617197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.721008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8da70-1bd8-47da-a99f-4abf2ea43ea6 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.728498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c62afb6-b8fa-418e-a78c-d87c0c452a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.729324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62afb6-b8fa-418e-a78c-d87c0c452a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.729711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62afb6-b8fa-418e-a78c-d87c0c452a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.730174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c62afb6-b8fa-418e-a78c-d87c0c452a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.731048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62afb6-b8fa-418e-a78c-d87c0c452a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.733257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c62afb6-b8fa-418e-a78c-d87c0c452a91 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.746657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 806/1600] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.748064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661f16b-e4cc-4eb1-825b-7f4e7e198598 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.748379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661f16b-e4cc-4eb1-825b-7f4e7e198598 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.748711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1661f16b-e4cc-4eb1-825b-7f4e7e198598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.780937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1661f16b-e4cc-4eb1-825b-7f4e7e198598 None None] MFA Rules not processed for user `38f9c3f296de43cea60525957baed325`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:52.782201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1661f16b-e4cc-4eb1-825b-7f4e7e198598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:52.829390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.830204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.830557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.831270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.832112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.833832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.854523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 795/1601] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.854523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.866709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09cb105-87f4-4602-be6e-c65da85e1eff admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.872498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2e8a8-6f48-4d88-96a0-a8c322706e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.873931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2e8a8-6f48-4d88-96a0-a8c322706e71 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.874573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2e8a8-6f48-4d88-96a0-a8c322706e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.874749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2e8a8-6f48-4d88-96a0-a8c322706e71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.875463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2e8a8-6f48-4d88-96a0-a8c322706e71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.878095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2e8a8-6f48-4d88-96a0-a8c322706e71 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.881968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 807/1602] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:52.886422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661f16b-e4cc-4eb1-825b-7f4e7e198598 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:52.894672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 796/1603] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.894672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b8c9-2aec-4409-a5a6-f94c1532c18e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.894672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b8c9-2aec-4409-a5a6-f94c1532c18e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.894672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9b8c9-2aec-4409-a5a6-f94c1532c18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.902522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1661f16b-e4cc-4eb1-825b-7f4e7e198598 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:52.909297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303afcef-cb3c-4299-897b-cbb6e497a5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:52.911094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303afcef-cb3c-4299-897b-cbb6e497a5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.911576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303afcef-cb3c-4299-897b-cbb6e497a5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.912078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303afcef-cb3c-4299-897b-cbb6e497a5dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.913053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303afcef-cb3c-4299-897b-cbb6e497a5dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:52.915349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303afcef-cb3c-4299-897b-cbb6e497a5dd admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:52.916169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6ed9b8c9-2aec-4409-a5a6-f94c1532c18e None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:52.917553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6ed9b8c9-2aec-4409-a5a6-f94c1532c18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:52.929279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 808/1604] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:52.930663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1a41-9c8d-41d3-8f45-07aca0e7af4a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:52.930988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1a41-9c8d-41d3-8f45-07aca0e7af4a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:52.931269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b1a41-9c8d-41d3-8f45-07aca0e7af4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:52.964792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ff1b1a41-9c8d-41d3-8f45-07aca0e7af4a None None] MFA Rules not processed for user `164ce06a5a1a4cf7bab104b1270c9de1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:52.967062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ff1b1a41-9c8d-41d3-8f45-07aca0e7af4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:52.975178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:53.043727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 797/1605] 10.222.0.42 () {58 vars in 1058 bytes} [Thu May 26 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:53.045272 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9b8c9-2aec-4409-a5a6-f94c1532c18e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:53.058377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:53.066523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9b8c9-2aec-4409-a5a6-f94c1532c18e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:53.074629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:53.076255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.076749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.077323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] PATH_INFO: `/v3/projects/c5f69588fe354d20b58594919deb4110/users/9b94748ea78342629281f2360a54bee4/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:53.099475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 809/1606] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:52 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:53.099475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b1a41-9c8d-41d3-8f45-07aca0e7af4a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:53.112817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5f69588fe354d20b58594919deb4110, user_id=9b94748ea78342629281f2360a54bee4, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:53.116590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b1a41-9c8d-41d3-8f45-07aca0e7af4a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:53.120159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:53.122756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:53.123853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.124217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.124771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] PATH_INFO: `/v3/projects/318999cb1f874176b2523de9686febc1/users/ee4307a7bdf84fe18efd6ba3dbe50c23/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:53.143180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=318999cb1f874176b2523de9686febc1, user_id=ee4307a7bdf84fe18efd6ba3dbe50c23, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:53.146189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:53.226894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 798/1607] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:53 2022] PUT /identity/v3/projects/c5f69588fe354d20b58594919deb4110/users/9b94748ea78342629281f2360a54bee4/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:53.228652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:53.303637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 810/1608] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:53 2022] PUT /identity/v3/projects/318999cb1f874176b2523de9686febc1/users/ee4307a7bdf84fe18efd6ba3dbe50c23/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:53.305142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:53.308662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb01cd36-9b2c-44af-88e8-c7f5e82f7490 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:53.402647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:53.403340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.403674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.403987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:53.404697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:53.406647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:53.414049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892073f8-8466-4182-951b-412fc111a3bb admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:53.422702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:53.424783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.425678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.426349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] PATH_INFO: `/v3/projects/6f94cb4b483c44869229b4b073c19d96/users/ddc77b5aa7c6444f8bead90df7b9bdf8/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:53.455856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f94cb4b483c44869229b4b073c19d96, user_id=ddc77b5aa7c6444f8bead90df7b9bdf8, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:53.460315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:53.604759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 799/1609] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:53 2022] POST /identity/v3/projects => generated 363 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:53.605552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:53.627056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 811/1610] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:53 2022] PUT /identity/v3/projects/6f94cb4b483c44869229b4b073c19d96/users/ddc77b5aa7c6444f8bead90df7b9bdf8/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:53.627971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:53.681291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:53.718695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b083e8f-ec00-480a-a9a0-b2922f09e0c8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:53.748926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fcbb88-69c8-41ea-98a7-4490ffcb11c2 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:53.757957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:53.759044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.759513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.759848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:53.821082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373fd42-b5b8-4d73-9c5b-3e3fb5dd4e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:53.822825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373fd42-b5b8-4d73-9c5b-3e3fb5dd4e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.824527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373fd42-b5b8-4d73-9c5b-3e3fb5dd4e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.824527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373fd42-b5b8-4d73-9c5b-3e3fb5dd4e8d admin admin] PATH_INFO: `/v3/projects/3969075d983246cf8c4c7a45a57d5338/users/6214705ae31e459891f7a215f45a4cb2/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:53.841210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:53.855617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373fd42-b5b8-4d73-9c5b-3e3fb5dd4e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3969075d983246cf8c4c7a45a57d5338, user_id=6214705ae31e459891f7a215f45a4cb2, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:53.861131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373fd42-b5b8-4d73-9c5b-3e3fb5dd4e8d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:53.866973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:53.871548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:53.888899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 812/1611] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:53.890051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:53.903550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af075e-c981-43c0-9fc9-dd2b1f17da1e service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:53.909015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1fd6c9-20fa-44db-bbdf-766435a0edd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:53.911101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fd6c9-20fa-44db-bbdf-766435a0edd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.911101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fd6c9-20fa-44db-bbdf-766435a0edd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.911101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1fd6c9-20fa-44db-bbdf-766435a0edd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:53.911732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1fd6c9-20fa-44db-bbdf-766435a0edd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:53.915623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1fd6c9-20fa-44db-bbdf-766435a0edd2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:53.984541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 800/1612] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:53 2022] PUT /identity/v3/projects/3969075d983246cf8c4c7a45a57d5338/users/6214705ae31e459891f7a215f45a4cb2/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:53.986339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672ab14-0b8c-44f3-b7cc-43576ef64924 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:53.986800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672ab14-0b8c-44f3-b7cc-43576ef64924 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:53.987263 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672ab14-0b8c-44f3-b7cc-43576ef64924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.011723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-9672ab14-0b8c-44f3-b7cc-43576ef64924 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:54.018734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9672ab14-0b8c-44f3-b7cc-43576ef64924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:54.057089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 813/1613] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:26:53 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:54.059526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91b4aa-2f0a-4cdc-a0b1-50d2e7bf3422 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.060163 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91b4aa-2f0a-4cdc-a0b1-50d2e7bf3422 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.060806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91b4aa-2f0a-4cdc-a0b1-50d2e7bf3422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.096172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4d91b4aa-2f0a-4cdc-a0b1-50d2e7bf3422 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:54.098645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4d91b4aa-2f0a-4cdc-a0b1-50d2e7bf3422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:54.146364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 801/1614] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:54.148723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672ab14-0b8c-44f3-b7cc-43576ef64924 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.170126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672ab14-0b8c-44f3-b7cc-43576ef64924 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.208363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 814/1615] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:54.209864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a27a95-77f7-4f89-857b-92a431d466af None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.210227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a27a95-77f7-4f89-857b-92a431d466af None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.210696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a27a95-77f7-4f89-857b-92a431d466af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.250240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-55a27a95-77f7-4f89-857b-92a431d466af None None] MFA Rules not processed for user `ddc77b5aa7c6444f8bead90df7b9bdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:54.251541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-55a27a95-77f7-4f89-857b-92a431d466af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:54.262660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.263367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.263933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.263933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.264775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.266795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.356878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 815/1616] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:54.358321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446ed6-6f77-4a13-8cf7-3487b9c1eec7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.358626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446ed6-6f77-4a13-8cf7-3487b9c1eec7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.359018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76446ed6-6f77-4a13-8cf7-3487b9c1eec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.368323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:54.376466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 802/1617] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:54 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:54.377748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.385020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-76446ed6-6f77-4a13-8cf7-3487b9c1eec7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:54.386376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-76446ed6-6f77-4a13-8cf7-3487b9c1eec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:54.391273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1131d74-e2b7-4bb4-a772-f84e3c8eece6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.398057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 816/1618] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:54.398921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76446ed6-6f77-4a13-8cf7-3487b9c1eec7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.402633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3be30-97b1-43c5-8470-f37b8ff569b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.403344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3be30-97b1-43c5-8470-f37b8ff569b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.403628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3be30-97b1-43c5-8470-f37b8ff569b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.404246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3be30-97b1-43c5-8470-f37b8ff569b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.404784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3be30-97b1-43c5-8470-f37b8ff569b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.408399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3be30-97b1-43c5-8470-f37b8ff569b9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.478287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:54.489123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 803/1619] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:26:54 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:26:54.491279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d1f1a-b934-41cb-8ad1-1139d281c5a1 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.491773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d1f1a-b934-41cb-8ad1-1139d281c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.492139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d1f1a-b934-41cb-8ad1-1139d281c5a1 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.493917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 804/1620] 10.222.0.42 () {54 vars in 912 bytes} [Thu May 26 21:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:26:54.494837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d1f1a-b934-41cb-8ad1-1139d281c5a1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.497272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76446ed6-6f77-4a13-8cf7-3487b9c1eec7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.502548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.503232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.504088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.504373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.509864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833d1f1a-b934-41cb-8ad1-1139d281c5a1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.513264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.513962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.514247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.514583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.515023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.516567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.527397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.529089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.529680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 817/1621] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:54.535158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.546247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 805/1622] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:54.547682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fe1dc-a788-4cf0-8eae-940c6facf050 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.547910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.555509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdc9bf6-589e-4e0b-8644-0dd5a2d7a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.556258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc9bf6-589e-4e0b-8644-0dd5a2d7a8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.556597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc9bf6-589e-4e0b-8644-0dd5a2d7a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.556882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdc9bf6-589e-4e0b-8644-0dd5a2d7a8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.558368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc9bf6-589e-4e0b-8644-0dd5a2d7a8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.560326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdc9bf6-589e-4e0b-8644-0dd5a2d7a8c4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.562354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39a16b-db3e-49cb-8dd2-7cf7713660b7 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.571504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cda301a-f694-4094-ae7d-77923c81c0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.572979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda301a-f694-4094-ae7d-77923c81c0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.573348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda301a-f694-4094-ae7d-77923c81c0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.573789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cda301a-f694-4094-ae7d-77923c81c0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.576393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 818/1623] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:54.578261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3270e65-f7b3-4028-a238-81381061f034 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.578597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3270e65-f7b3-4028-a238-81381061f034 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.578928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3270e65-f7b3-4028-a238-81381061f034 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.581613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 819/1624] 10.222.0.42 () {54 vars in 961 bytes} [Thu May 26 21:26:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:26:54.583936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3270e65-f7b3-4028-a238-81381061f034 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.593167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda301a-f694-4094-ae7d-77923c81c0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.595058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cda301a-f694-4094-ae7d-77923c81c0e8 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.602045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3270e65-f7b3-4028-a238-81381061f034 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.607616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 806/1625] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:54.608198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.609069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.609504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.609810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.610596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.613467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.614511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e9723-dd79-4dcc-ae4d-e2077cffc1a1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.615040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e9723-dd79-4dcc-ae4d-e2077cffc1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.615678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e9723-dd79-4dcc-ae4d-e2077cffc1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.633444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 820/1626] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:54.633444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.641347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-a74e9723-dd79-4dcc-ae4d-e2077cffc1a1 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:54.642894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a74e9723-dd79-4dcc-ae4d-e2077cffc1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:54.646905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863e257-99b7-44e5-9e45-5618a5f5516a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.653294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.656910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.656910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.656910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] PATH_INFO: `/v3/projects/0e961999a82e43b282ab8f84ba1b6d76/users/b5f008d5429c455e842d3720f1a95b8d/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.677330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e961999a82e43b282ab8f84ba1b6d76, user_id=b5f008d5429c455e842d3720f1a95b8d, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.694318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.744327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 807/1627] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:54.746064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9467a-67ab-4484-878a-fe3e6aa3fbd3 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.746863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9467a-67ab-4484-878a-fe3e6aa3fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.747352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9467a-67ab-4484-878a-fe3e6aa3fbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.774106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f4a9467a-67ab-4484-878a-fe3e6aa3fbd3 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:54.775560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f4a9467a-67ab-4484-878a-fe3e6aa3fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:54.791617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 808/1628] 10.222.0.42 () {58 vars in 1058 bytes} [Thu May 26 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:54.795555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9467a-67ab-4484-878a-fe3e6aa3fbd3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.805424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9467a-67ab-4484-878a-fe3e6aa3fbd3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.811209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.813043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.813423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.813771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] PATH_INFO: `/v3/projects/263357827d4f4dad8741c45190682d51/users/190a1c87b9d14faab257bce02b535575/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:54.836245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=263357827d4f4dad8741c45190682d51, user_id=190a1c87b9d14faab257bce02b535575, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:54.839304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:54.874397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 821/1629] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:54 2022] PUT /identity/v3/projects/0e961999a82e43b282ab8f84ba1b6d76/users/b5f008d5429c455e842d3720f1a95b8d/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:54.875284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:54.958017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9ddbfb-2fca-4754-bfe5-4a216d024ad2 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:54.966679 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf1c26-1bbb-433b-8e44-27713bcb8ddf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:54.968325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf1c26-1bbb-433b-8e44-27713bcb8ddf service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:54.975008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf1c26-1bbb-433b-8e44-27713bcb8ddf service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:54.975008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf1c26-1bbb-433b-8e44-27713bcb8ddf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.015544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 809/1630] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:54 2022] PUT /identity/v3/projects/263357827d4f4dad8741c45190682d51/users/190a1c87b9d14faab257bce02b535575/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:55.017050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:55.124969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf1c26-1bbb-433b-8e44-27713bcb8ddf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:55.127397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf1c26-1bbb-433b-8e44-27713bcb8ddf service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:55.143504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3de059-95ce-46ef-84ed-c83c39c103ec admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:55.150375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 822/1631] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:55.150776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:55.151615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.152095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.152447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.153233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c71cf-fe3a-4f1c-92eb-1ac951041e38 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.154063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c71cf-fe3a-4f1c-92eb-1ac951041e38 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.154627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c71cf-fe3a-4f1c-92eb-1ac951041e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.169198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:55.208971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-389c71cf-fe3a-4f1c-92eb-1ac951041e38 None None] MFA Rules not processed for user `71302686cd59424ba952b49c0fad3577`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:55.211048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-389c71cf-fe3a-4f1c-92eb-1ac951041e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:55.257081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:55.260758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:55.269904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:55.297215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 810/1632] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:55.298221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:55.394876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 823/1633] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:55.394876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c71cf-fe3a-4f1c-92eb-1ac951041e38 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:55.422923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1fec5-e0d5-4a29-9658-3b536133b055 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:55.427947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db5edb-e402-4e5a-96cc-442b48c0ac09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:55.428657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db5edb-e402-4e5a-96cc-442b48c0ac09 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.428945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db5edb-e402-4e5a-96cc-442b48c0ac09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.429226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db5edb-e402-4e5a-96cc-442b48c0ac09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.430303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db5edb-e402-4e5a-96cc-442b48c0ac09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:55.432076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5db5edb-e402-4e5a-96cc-442b48c0ac09 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:55.449201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 811/1634] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:55.451047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b163bf-d25d-490a-9df1-a2cb48f0428c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.451344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b163bf-d25d-490a-9df1-a2cb48f0428c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.453855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b163bf-d25d-490a-9df1-a2cb48f0428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.489472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c4b163bf-d25d-490a-9df1-a2cb48f0428c None None] MFA Rules not processed for user `71302686cd59424ba952b49c0fad3577`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:55.491916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c4b163bf-d25d-490a-9df1-a2cb48f0428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:55.520039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 812/1635] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:55.521155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b163bf-d25d-490a-9df1-a2cb48f0428c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:55.554448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b163bf-d25d-490a-9df1-a2cb48f0428c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:55.574908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c71cf-fe3a-4f1c-92eb-1ac951041e38 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:55.577855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:55.579015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.579426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.579951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] PATH_INFO: `/v3/projects/0e961999a82e43b282ab8f84ba1b6d76/users/b5f008d5429c455e842d3720f1a95b8d/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.582884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e961999a82e43b282ab8f84ba1b6d76, user_id=b5f008d5429c455e842d3720f1a95b8d, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:55.586328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:55.586826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac405c2-df36-4b3d-8c96-009277074e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:55.588582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac405c2-df36-4b3d-8c96-009277074e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.591724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac405c2-df36-4b3d-8c96-009277074e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.592536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac405c2-df36-4b3d-8c96-009277074e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.594095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac405c2-df36-4b3d-8c96-009277074e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:55.598660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac405c2-df36-4b3d-8c96-009277074e45 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:55.625023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 813/1636] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:55 2022] PUT /identity/v3/projects/0e961999a82e43b282ab8f84ba1b6d76/users/b5f008d5429c455e842d3720f1a95b8d/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:55.625670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 824/1637] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:26:55 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:26:55.636817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:55.638744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3ead-0195-48c9-85a5-c38e168b1b96 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.639050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3ead-0195-48c9-85a5-c38e168b1b96 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.639330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa3ead-0195-48c9-85a5-c38e168b1b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.667956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-aefa3ead-0195-48c9-85a5-c38e168b1b96 None None] MFA Rules not processed for user `b5f008d5429c455e842d3720f1a95b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:55.669506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-aefa3ead-0195-48c9-85a5-c38e168b1b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:55.774004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c3d31-f040-4f79-b2b6-df19f7b6eda6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:55.789884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902d278-9bdd-4c78-bdf5-4787e214680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:55.789884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902d278-9bdd-4c78-bdf5-4787e214680d admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.789884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902d278-9bdd-4c78-bdf5-4787e214680d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.789884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902d278-9bdd-4c78-bdf5-4787e214680d admin admin] PATH_INFO: `/v3/projects/263357827d4f4dad8741c45190682d51/users/190a1c87b9d14faab257bce02b535575/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.793017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 825/1638] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:55.798421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902d278-9bdd-4c78-bdf5-4787e214680d admin admin] RBAC: Authorizing `identity:create_grant(project_id=263357827d4f4dad8741c45190682d51, user_id=190a1c87b9d14faab257bce02b535575, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:55.799015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa3ead-0195-48c9-85a5-c38e168b1b96 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:55.803444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902d278-9bdd-4c78-bdf5-4787e214680d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:55.841193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 814/1639] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:26:55 2022] PUT /identity/v3/projects/263357827d4f4dad8741c45190682d51/users/190a1c87b9d14faab257bce02b535575/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:26:55.843675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e5b1f-d057-4784-9e90-805a6ff45bcc None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.846293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e5b1f-d057-4784-9e90-805a6ff45bcc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.847516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92e5b1f-d057-4784-9e90-805a6ff45bcc None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.855093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 815/1640] 10.222.0.42 () {54 vars in 912 bytes} [Thu May 26 21:26:55 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:26:55.855093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121589c0-292c-45b7-a7a0-bf0b6470dd47 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:55.855093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121589c0-292c-45b7-a7a0-bf0b6470dd47 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:55.855093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121589c0-292c-45b7-a7a0-bf0b6470dd47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:55.912249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa3ead-0195-48c9-85a5-c38e168b1b96 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:55.920629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:55.932905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-121589c0-292c-45b7-a7a0-bf0b6470dd47 None None] MFA Rules not processed for user `190a1c87b9d14faab257bce02b535575`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:55.935157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-121589c0-292c-45b7-a7a0-bf0b6470dd47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:55.954231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:56.014206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:56.015908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:56.016358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:56.017033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:56.062999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 816/1641] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:56.065703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cedce-9b6d-4aaf-8719-ebe16a598acf None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:56.066216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cedce-9b6d-4aaf-8719-ebe16a598acf None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:56.066624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260cedce-9b6d-4aaf-8719-ebe16a598acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:56.096334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-260cedce-9b6d-4aaf-8719-ebe16a598acf None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:56.100451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-260cedce-9b6d-4aaf-8719-ebe16a598acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:56.153107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:56.155344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:56.171815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 826/1642] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:26:56.189662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 817/1643] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:56.324061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:56.335718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6b6e80-51b2-4ca3-8b0d-162f88fdbb63 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:56.342893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:56.346096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:56.346096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:56.346096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:56.360633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:56.362325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:56.376907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 827/1644] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:56 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:56.545509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cedce-9b6d-4aaf-8719-ebe16a598acf None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:56.563151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260cedce-9b6d-4aaf-8719-ebe16a598acf None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:56.571831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:56.572886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:56.573494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:56.573970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:56.600339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:56.602342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:56.620850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 818/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:57.167644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:57.194392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cec3ca-81af-4bb5-a1c2-f5aca6d7baad service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:57.199132 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb049867-33a7-42bf-b110-9d008bcdbe64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:57.205957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:57.232588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049867-33a7-42bf-b110-9d008bcdbe64 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:57.239580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049867-33a7-42bf-b110-9d008bcdbe64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:57.239580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb049867-33a7-42bf-b110-9d008bcdbe64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:57.267044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfed062-d3ff-41a2-86f1-a0f1040c66ec service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:57.304037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3681a28e-6d2e-4ee1-93b0-c82c84f3b2f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:57.307141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681a28e-6d2e-4ee1-93b0-c82c84f3b2f2 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:57.310267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681a28e-6d2e-4ee1-93b0-c82c84f3b2f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:57.311167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3681a28e-6d2e-4ee1-93b0-c82c84f3b2f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:57.473870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681a28e-6d2e-4ee1-93b0-c82c84f3b2f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:57.477069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3681a28e-6d2e-4ee1-93b0-c82c84f3b2f2 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:57.487337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049867-33a7-42bf-b110-9d008bcdbe64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:57.495270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb049867-33a7-42bf-b110-9d008bcdbe64 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:57.516151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 819/1646] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:57.543881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 828/1647] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:57 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:58.337326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a0b8f-c4a3-4999-8543-1627c66024e2 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:58.337947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a0b8f-c4a3-4999-8543-1627c66024e2 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:58.337947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a0b8f-c4a3-4999-8543-1627c66024e2 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:58.339388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 820/1648] 10.222.0.42 () {54 vars in 912 bytes} [Thu May 26 21:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:26:58.345629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991d4de-97df-4f88-8485-c699064ae87d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:58.346207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991d4de-97df-4f88-8485-c699064ae87d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:58.346290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991d4de-97df-4f88-8485-c699064ae87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:58.427516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3991d4de-97df-4f88-8485-c699064ae87d None None] MFA Rules not processed for user `e757ff94b7474fd99a1c7221be19c64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:26:58.432100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3991d4de-97df-4f88-8485-c699064ae87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:26:58.580051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:26:58.658888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 829/1649] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:26:58.749171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a0b8f-c4a3-4999-8543-1627c66024e2 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:58.778926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a0b8f-c4a3-4999-8543-1627c66024e2 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:58.796118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175920d-7b59-4866-9bae-b610d81dbd63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:58.796816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175920d-7b59-4866-9bae-b610d81dbd63 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:58.797124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175920d-7b59-4866-9bae-b610d81dbd63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:58.797401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175920d-7b59-4866-9bae-b610d81dbd63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:58.822517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175920d-7b59-4866-9bae-b610d81dbd63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:58.824040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175920d-7b59-4866-9bae-b610d81dbd63 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:26:58.854338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 821/1650] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:26:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:26:59.668866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991d4de-97df-4f88-8485-c699064ae87d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:26:59.903566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991d4de-97df-4f88-8485-c699064ae87d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:26:59.903566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:26:59.903566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:26:59.903566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:26:59.904013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] PATH_INFO: `/v3/users/164ce06a5a1a4cf7bab104b1270c9de1` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:26:59.906695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=164ce06a5a1a4cf7bab104b1270c9de1)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:26:59.909244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: WARNING py.warnings [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:26:59.909244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: warnings.warn(msg) May 26 21:26:59.909244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: May 26 21:26:59.916826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:00.044422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c112c9-1d04-4b1b-a155-6488c4e9de0f None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:00.044422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c112c9-1d04-4b1b-a155-6488c4e9de0f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:00.044422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c112c9-1d04-4b1b-a155-6488c4e9de0f None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:00.044422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 822/1651] 10.222.0.42 () {54 vars in 960 bytes} [Thu May 26 21:27:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:27:00.044422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280d788-7b9c-4206-88d7-d90f9dd3c072 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:00.044422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280d788-7b9c-4206-88d7-d90f9dd3c072 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:00.045153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280d788-7b9c-4206-88d7-d90f9dd3c072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:00.078669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-4280d788-7b9c-4206-88d7-d90f9dd3c072 None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:00.084182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4280d788-7b9c-4206-88d7-d90f9dd3c072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:00.098235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:00.194098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 823/1652] 10.222.0.42 () {58 vars in 1057 bytes} [Thu May 26 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:00.205844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280d788-7b9c-4206-88d7-d90f9dd3c072 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:00.237053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280d788-7b9c-4206-88d7-d90f9dd3c072 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:00.249059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee78ecf-5a3a-4022-a853-3a5841b43906 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:00.277031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee78ecf-5a3a-4022-a853-3a5841b43906 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:00.278733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee78ecf-5a3a-4022-a853-3a5841b43906 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:00.280950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee78ecf-5a3a-4022-a853-3a5841b43906 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:00.295239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-aee78ecf-5a3a-4022-a853-3a5841b43906 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:00.326066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee78ecf-5a3a-4022-a853-3a5841b43906 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:00.329601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee78ecf-5a3a-4022-a853-3a5841b43906 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:00.358266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 824/1653] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:00.445883 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164ce06a5a1a4cf7bab104b1270c9de1'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:00.512566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164ce06a5a1a4cf7bab104b1270c9de1'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:00.620215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164ce06a5a1a4cf7bab104b1270c9de1'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:00.621916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164ce06a5a1a4cf7bab104b1270c9de1'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:00.626353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164ce06a5a1a4cf7bab104b1270c9de1'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:00.775175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 830/1654] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:26:59 2022] DELETE /identity/v3/users/164ce06a5a1a4cf7bab104b1270c9de1 => generated 0 bytes in 1102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:00.781580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:00.782510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:00.782947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:00.858762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:00.871566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:01.367908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 825/1655] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 587 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:01.396446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:01.398297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:01.625128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca6a10-75a4-4e3b-8d1c-833eae4a0c21 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:01.637720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:01.638821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:01.639421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:01.639851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:01.677009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:01.690093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:01.738293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 831/1656] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:04.613171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:04.615174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:04.639537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d8d-934a-4e94-ab98-9695ec77f1de None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:04.653600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:04.654775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:04.655198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:04.655602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] PATH_INFO: `/v3/projects/30a7e028890f440780a841bac629a2eb` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:04.661922 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a7e028890f440780a841bac629a2eb)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:04.663004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: WARNING py.warnings [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 21:27:04.663004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: warnings.warn(msg) May 26 21:27:04.663004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: May 26 21:27:04.664250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:04.897491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a7e028890f440780a841bac629a2eb', 'request_id': 'req-5b0d6ede-0485-4f8d-9c20-0615c7de8499'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:04.939715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a7e028890f440780a841bac629a2eb', 'request_id': 'req-5b0d6ede-0485-4f8d-9c20-0615c7de8499'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:04.956011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a7e028890f440780a841bac629a2eb', 'request_id': 'req-5b0d6ede-0485-4f8d-9c20-0615c7de8499'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.130145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] The token cache is being invalidate because project 30a7e028890f440780a841bac629a2eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:05.130747 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.133078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5b0d6ede-0485-4f8d-9c20-0615c7de8499 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.141156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 826/1657] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:04 2022] DELETE /identity/v3/projects/30a7e028890f440780a841bac629a2eb => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:05.142399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:05.169451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:05.183219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418df4db-eab9-45b3-a6bf-254510fb32a1 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:05.350770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:05.352549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:05.353031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:05.356901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:05.358125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:05.358611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:05.363156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] PATH_INFO: `/v3/users/190a1c87b9d14faab257bce02b535575` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:05.363156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=190a1c87b9d14faab257bce02b535575)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:05.368655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:05.420905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] MFA Rules not processed for user `9b94748ea78342629281f2360a54bee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:05.423234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:05.437613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:05.598122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 827/1658] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:05.614220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '190a1c87b9d14faab257bce02b535575'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.615020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:05.616925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:05.666276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '190a1c87b9d14faab257bce02b535575'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.765697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd5a1f-4863-4cad-8bce-95f77b21f968 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:05.776993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:05.777846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:05.778220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:05.778588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:05.790063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:05.792458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:05.802131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '190a1c87b9d14faab257bce02b535575'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.802819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '190a1c87b9d14faab257bce02b535575'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.803687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '190a1c87b9d14faab257bce02b535575'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:05.807910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 828/1659] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:05.822715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 832/1660] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:05 2022] DELETE /identity/v3/users/190a1c87b9d14faab257bce02b535575 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:06.652367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:06.654069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:06.678684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd4b814-b964-42f4-a27d-ab528c3041f4 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:06.697423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:06.700164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:06.930524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d46f-1277-40c1-a580-f9863e5fe6b9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:06.945387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:06.946721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:06.947334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:06.949862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] PATH_INFO: `/v3/projects/263357827d4f4dad8741c45190682d51` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:06.958439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=263357827d4f4dad8741c45190682d51)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:06.959698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:06.985550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:06.986183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:06.993189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:06.993189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] PATH_INFO: `/v3/users/ddc77b5aa7c6444f8bead90df7b9bdf8` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:07.007473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc77b5aa7c6444f8bead90df7b9bdf8)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:07.016496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:07.328057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:07.352765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc77b5aa7c6444f8bead90df7b9bdf8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.380680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc77b5aa7c6444f8bead90df7b9bdf8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.413288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:07.505162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc77b5aa7c6444f8bead90df7b9bdf8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.505974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc77b5aa7c6444f8bead90df7b9bdf8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.506715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5cc48395-ab1b-41f7-8cd2-ae8aefcbaffb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc77b5aa7c6444f8bead90df7b9bdf8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.538048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 829/1661] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:06 2022] DELETE /identity/v3/users/ddc77b5aa7c6444f8bead90df7b9bdf8 => generated 0 bytes in 886 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:07.544464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:07.544999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:07.545700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:07.575443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:07.577566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:07.605788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '263357827d4f4dad8741c45190682d51', 'request_id': 'req-8cf8accd-ae18-4b61-b065-df8fab544d7f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.617941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '263357827d4f4dad8741c45190682d51', 'request_id': 'req-8cf8accd-ae18-4b61-b065-df8fab544d7f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.632754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '263357827d4f4dad8741c45190682d51', 'request_id': 'req-8cf8accd-ae18-4b61-b065-df8fab544d7f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.684149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 830/1662] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:07.717609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:07.719595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:07.736081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] The token cache is being invalidate because project 263357827d4f4dad8741c45190682d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:07.736081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.736750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8cf8accd-ae18-4b61-b065-df8fab544d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:07.738436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 833/1663] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:06 2022] DELETE /identity/v3/projects/263357827d4f4dad8741c45190682d51 => generated 0 bytes in 1042 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:07.743995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd201402-f0d2-4d5f-97b7-1461f2f52843 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:07.744288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd201402-f0d2-4d5f-97b7-1461f2f52843 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:07.744570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd201402-f0d2-4d5f-97b7-1461f2f52843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:07.795775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-bd201402-f0d2-4d5f-97b7-1461f2f52843 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:07.797080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bd201402-f0d2-4d5f-97b7-1461f2f52843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:07.856512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 834/1664] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:07.864687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da753c6-9440-47bb-905c-d76edc4e4f56 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:07.865100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da753c6-9440-47bb-905c-d76edc4e4f56 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:07.865477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da753c6-9440-47bb-905c-d76edc4e4f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:07.900019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46334f19-8145-47ad-89ff-11ead7935bc1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:07.914056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:07.915075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:07.916252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:07.916980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:07.921082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4da753c6-9440-47bb-905c-d76edc4e4f56 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:07.923220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4da753c6-9440-47bb-905c-d76edc4e4f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:07.923355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:07.936182 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 835/1665] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:07.940223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da753c6-9440-47bb-905c-d76edc4e4f56 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:07.953541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:07.955338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:07.981543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 831/1666] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:07.985328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da753c6-9440-47bb-905c-d76edc4e4f56 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:07.992862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7842c-90ed-4285-82b1-8c29e1c83580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:07.993925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7842c-90ed-4285-82b1-8c29e1c83580 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:07.996595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7842c-90ed-4285-82b1-8c29e1c83580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:07.997154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7842c-90ed-4285-82b1-8c29e1c83580 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:07.998562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7842c-90ed-4285-82b1-8c29e1c83580 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:07.999070 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7842c-90ed-4285-82b1-8c29e1c83580 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:08.004808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7842c-90ed-4285-82b1-8c29e1c83580 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:08.054211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 836/1667] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:27:08.058537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:08.071521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c504-cf79-4e07-aac5-c115a2342574 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:08.084781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0db7c-770f-48b8-967d-9064745ec372 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:08.084781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0db7c-770f-48b8-967d-9064745ec372 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:08.084781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc0db7c-770f-48b8-967d-9064745ec372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:08.104445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:08.104580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:08.104866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:08.105142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:08.105839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:08.116099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:08.139719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ddc0db7c-770f-48b8-967d-9064745ec372 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:08.141601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ddc0db7c-770f-48b8-967d-9064745ec372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:08.266660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 832/1668] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:08 2022] POST /identity/v3/projects => generated 365 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:08.271397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:08.365243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 837/1669] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:27:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:08.383189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:08.437852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093bc4f5-1f43-4ed0-8b73-fd289c3be68a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:08.447445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:08.448402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:08.448691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:08.448972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:08.450071 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:08.457378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:08.591867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 833/1670] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:08 2022] POST /identity/v3/users => generated 283 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:08.595858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0db7c-770f-48b8-967d-9064745ec372 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:08.615339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc0db7c-770f-48b8-967d-9064745ec372 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:08.626441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:08.631304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:08.631687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:08.632001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:08.632853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:08.635234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:08.665924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 838/1671] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:08.669876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:08.683479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ba24c-6102-46ab-b05c-aaed19d410ab admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:08.705073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:08.709952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:08.709952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:08.709952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:08.710708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:08.712397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:08.725950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 834/1672] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:08.729849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:08.741137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a41c9-1832-4696-894a-e4670ccd3eac admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:08.747463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:08.748481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:08.748826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:08.749164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] PATH_INFO: `/v3/projects/17ce743073c74a6bb4b0b9dbb651a20f/users/5c7ebce951bf40e3b7eaa2693abfdae4/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:08.767699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ce743073c74a6bb4b0b9dbb651a20f, user_id=5c7ebce951bf40e3b7eaa2693abfdae4, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:08.772053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:08.794078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:08.796041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:08.840461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457416f-142d-4cb2-a2ad-24e85133497f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:08.857121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 839/1673] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:08 2022] PUT /identity/v3/projects/17ce743073c74a6bb4b0b9dbb651a20f/users/5c7ebce951bf40e3b7eaa2693abfdae4/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:08.866313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:08.868507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:09.040927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:09.042627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:09.043077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:09.044026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] PATH_INFO: `/v3/projects/6f94cb4b483c44869229b4b073c19d96` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:09.052508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:09.057918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f94cb4b483c44869229b4b073c19d96)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:09.060217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:09.086013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dbb16c-70bd-4b4f-a47a-355d6ab0cf88 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:09.099979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96197de0-df48-4dbc-aa3e-8b4be6e39da4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:09.100681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96197de0-df48-4dbc-aa3e-8b4be6e39da4 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:09.104195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96197de0-df48-4dbc-aa3e-8b4be6e39da4 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:09.104195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96197de0-df48-4dbc-aa3e-8b4be6e39da4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:09.211006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96197de0-df48-4dbc-aa3e-8b4be6e39da4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:09.212753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96197de0-df48-4dbc-aa3e-8b4be6e39da4 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:09.227592 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 840/1674] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:27:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:27:09.233049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:09.233356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:09.233645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:09.238651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f94cb4b483c44869229b4b073c19d96', 'request_id': 'req-0a388658-fbed-4515-bbbc-3e32ca900c59'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:09.260496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f94cb4b483c44869229b4b073c19d96', 'request_id': 'req-0a388658-fbed-4515-bbbc-3e32ca900c59'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:09.277374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f94cb4b483c44869229b4b073c19d96', 'request_id': 'req-0a388658-fbed-4515-bbbc-3e32ca900c59'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:09.304772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] MFA Rules not processed for user `5c7ebce951bf40e3b7eaa2693abfdae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:09.306120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:09.429649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] The token cache is being invalidate because project 6f94cb4b483c44869229b4b073c19d96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:09.429948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:09.431447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0a388658-fbed-4515-bbbc-3e32ca900c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:09.438129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 835/1675] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:08 2022] DELETE /identity/v3/projects/6f94cb4b483c44869229b4b073c19d96 => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:09.443331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d44bd-fcbd-4ffd-80c3-2e3126ddc9cc None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:09.443658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d44bd-fcbd-4ffd-80c3-2e3126ddc9cc None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:09.443966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d44bd-fcbd-4ffd-80c3-2e3126ddc9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:09.499397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 841/1676] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:09.507981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:09.514175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-743d44bd-fcbd-4ffd-80c3-2e3126ddc9cc None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:09.516152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:09.519133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-743d44bd-fcbd-4ffd-80c3-2e3126ddc9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:09.670057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 836/1677] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:09.679047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8594ca-efb6-4b4a-8f2f-d32b1c57961c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:09.679879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8594ca-efb6-4b4a-8f2f-d32b1c57961c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:09.680312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8594ca-efb6-4b4a-8f2f-d32b1c57961c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:09.715520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1b8594ca-efb6-4b4a-8f2f-d32b1c57961c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:09.717650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1b8594ca-efb6-4b4a-8f2f-d32b1c57961c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:09.729238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d74472c-1e25-434d-9377-102b3564e3dd None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:09.743081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 837/1678] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:09.747544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8594ca-efb6-4b4a-8f2f-d32b1c57961c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:09.752434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:09.756401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:09.756401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:09.756401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:09.756401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:09.762133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:09.773839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8594ca-efb6-4b4a-8f2f-d32b1c57961c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:09.807459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:09.812852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:09.816087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:09.816424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:09.816704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:09.817478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:09.817791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:09.832859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:09.904658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 838/1679] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:27:09.906432 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:09.917176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 842/1680] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:09 2022] POST /identity/v3/projects => generated 365 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:09.918275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:09.938805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4fe938-07a7-4820-b2d2-8d95b07a6d4f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:10.116540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8e329-6121-40b1-aaa1-af42e57b2fe8 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:10.133826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:10.134930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:10.134930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:10.134930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:10.135652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:10.137519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:10.138323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:10.138626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:10.138976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:10.139925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:10.141532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:10.145294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:10.530183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:10.549087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 839/1681] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:09 2022] POST /identity/v3/projects => generated 367 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:10.549087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:10.550746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:10.763054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889c421-40ef-4318-803e-14968227a3bf admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:10.778553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:10.779729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:10.780195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:10.780610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] PATH_INFO: `/v3/users/5c66120fd1124221b4a5552ddb6cd145` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:10.797180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c66120fd1124221b4a5552ddb6cd145)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:10.799304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:10.886703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 843/1682] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:09 2022] POST /identity/v3/users => generated 289 bytes in 969 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:10.889289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:10.935276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9d2c58-c461-4d85-a713-e199fbd0c09f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:10.955802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:10.957755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:10.965145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:10.972208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:10.972208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:10.972208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:11.164043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c66120fd1124221b4a5552ddb6cd145'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:11.208877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 844/1683] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:10 2022] POST /identity/v3/users => generated 284 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:11.211415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:11.237318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c66120fd1124221b4a5552ddb6cd145'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:11.271473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac5da9-9e07-46d5-b8d7-e9ed42189122 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:11.281149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:11.283009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:11.283897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:11.284373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:11.288308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:11.302046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:11.319291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 845/1684] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:11.320748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:11.323164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c66120fd1124221b4a5552ddb6cd145'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:11.324139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c66120fd1124221b4a5552ddb6cd145'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:11.325469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c66120fd1124221b4a5552ddb6cd145'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:11.336568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b9425b-ecad-4a3a-a278-0ef58afef557 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:11.352733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cdecce-cae4-475a-bcfd-039aa698f5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:11.356252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdecce-cae4-475a-bcfd-039aa698f5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:11.358572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdecce-cae4-475a-bcfd-039aa698f5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:11.359657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cdecce-cae4-475a-bcfd-039aa698f5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:11.363735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdecce-cae4-475a-bcfd-039aa698f5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:11.369248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 840/1685] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:10 2022] DELETE /identity/v3/users/5c66120fd1124221b4a5552ddb6cd145 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:11.370171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cdecce-cae4-475a-bcfd-039aa698f5ff admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:11.370294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:11.373180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:11.394799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 846/1686] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:11.397244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:11.397728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:11.398548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:11.438020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:11.442580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:11.613627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 847/1687] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:11.616936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:11.616936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:11.621552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:11.634145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781c111-c0d6-45f9-be52-024fa50a0e81 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:11.654043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:11.654988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:11.655275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:11.655571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] PATH_INFO: `/v3/projects/1fd5b62c2f594934810880d6dbbd0c15/users/5bb8360f089e4983aea544a141f782e3/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:11.657547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27394d8d-8a61-471c-bfae-6c3ef1a2e37d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:11.671558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:11.673405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:11.673827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:11.674077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:11.674857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:11.677266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:11.700981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd5b62c2f594934810880d6dbbd0c15, user_id=5bb8360f089e4983aea544a141f782e3, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:11.705644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:11.716426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 848/1688] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:11.719562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:11.740470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:11.804099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 841/1689] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:11 2022] PUT /identity/v3/projects/1fd5b62c2f594934810880d6dbbd0c15/users/5bb8360f089e4983aea544a141f782e3/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:11.810663 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:11.896707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d7929-6484-4a36-ad1a-052c53e11ff8 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:11.917399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:12.031444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48069a-810a-431a-a2da-176889933a8a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:12.050176 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb6056-1ba7-449c-9255-f0ebcf7680b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:12.052060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6056-1ba7-449c-9255-f0ebcf7680b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:12.052729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6056-1ba7-449c-9255-f0ebcf7680b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:12.054131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6056-1ba7-449c-9255-f0ebcf7680b8 admin admin] PATH_INFO: `/v3/projects/9598c083dc2945f698e6c0f561f121bb/users/11ed6e0a382348528286aec0a2c6b2cc/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:12.077417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:12.080742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:12.080742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:12.080742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:12.118087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb6056-1ba7-449c-9255-f0ebcf7680b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9598c083dc2945f698e6c0f561f121bb, user_id=11ed6e0a382348528286aec0a2c6b2cc, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:12.131506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:12.136706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb6056-1ba7-449c-9255-f0ebcf7680b8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:12.138982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:12.171981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 849/1690] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:12.172933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:12.188850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51748f06-c647-4148-968d-f31f9278be2f service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:12.202811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:12.203733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:12.204463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:12.204915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:12.217849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:12.225626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:12.256572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 850/1691] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:12.293612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 842/1692] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:11 2022] PUT /identity/v3/projects/9598c083dc2945f698e6c0f561f121bb/users/11ed6e0a382348528286aec0a2c6b2cc/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:12.293704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:12.295046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:12.295337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:12.295634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:12.338086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbea580-c6fd-448a-aeb8-0e6882ae3be5 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:12.421559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] MFA Rules not processed for user `11ed6e0a382348528286aec0a2c6b2cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:12.422936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:12.567094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:12.604982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04861e97-e514-462c-830e-cc7e3e32837e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:12.606195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861e97-e514-462c-830e-cc7e3e32837e admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:12.610531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861e97-e514-462c-830e-cc7e3e32837e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:12.623195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04861e97-e514-462c-830e-cc7e3e32837e admin admin] PATH_INFO: `/v3/projects/1fd5b62c2f594934810880d6dbbd0c15/users/5bb8360f089e4983aea544a141f782e3/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:12.634535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04861e97-e514-462c-830e-cc7e3e32837e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd5b62c2f594934810880d6dbbd0c15, user_id=5bb8360f089e4983aea544a141f782e3, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:12.640840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04861e97-e514-462c-830e-cc7e3e32837e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:12.691979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 851/1693] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:12 2022] PUT /identity/v3/projects/1fd5b62c2f594934810880d6dbbd0c15/users/5bb8360f089e4983aea544a141f782e3/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:12.712559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:12.715676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:12.715676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:12.759111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 843/1694] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:12.774288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:12.784250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:12.812104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] MFA Rules not processed for user `5bb8360f089e4983aea544a141f782e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:12.814488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:12.932018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2594ad9a-315a-4e2d-83f9-09eabdc1a782 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:13.059648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:13.060359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:13.060648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:13.060932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:13.062986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 852/1695] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:13.078396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:13.079951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:13.100613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d43141-88c8-44c0-9945-b6fbf11a749c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:13.114207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:13.120223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:13.120223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:13.120223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:13.162053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:13.171695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:13.206708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:13.208967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:13.218172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 853/1696] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:13 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:13.232527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 844/1697] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:13.239729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:13.259396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:13.280325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:13.491929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0cbee-e2db-4a18-b435-1f00608f2b78 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:13.494072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:13.503981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b52225-05a6-42c1-bf01-76788c7cdc45 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:13.505385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:13.511630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:13.512056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:13.512499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] PATH_INFO: `/v3/projects/a3c67e60a5274e179ffd719af7cc9457` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:13.517758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c67e60a5274e179ffd719af7cc9457)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:13.523455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:13.523700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20e649a-ffa5-47fa-addc-e973983edfcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:13.530622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e649a-ffa5-47fa-addc-e973983edfcf service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:13.530996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e649a-ffa5-47fa-addc-e973983edfcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:13.531284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20e649a-ffa5-47fa-addc-e973983edfcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:13.544020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b20e649a-ffa5-47fa-addc-e973983edfcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:13.613730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:13.677280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c67e60a5274e179ffd719af7cc9457', 'request_id': 'req-6aa9a72a-1350-449e-ba4d-37a4247ee39a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:13.678861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e649a-ffa5-47fa-addc-e973983edfcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:13.694989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20e649a-ffa5-47fa-addc-e973983edfcf service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:13.705897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 845/1698] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:13.741868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c67e60a5274e179ffd719af7cc9457', 'request_id': 'req-6aa9a72a-1350-449e-ba4d-37a4247ee39a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:13.766164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c67e60a5274e179ffd719af7cc9457', 'request_id': 'req-6aa9a72a-1350-449e-ba4d-37a4247ee39a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:13.905815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] The token cache is being invalidate because project a3c67e60a5274e179ffd719af7cc9457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:13.906442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:13.917688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6aa9a72a-1350-449e-ba4d-37a4247ee39a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:13.920459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 854/1699] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:13 2022] DELETE /identity/v3/projects/a3c67e60a5274e179ffd719af7cc9457 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:13.925410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e894b86-8f86-4e9e-8278-107f78dc22bd None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:13.925710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e894b86-8f86-4e9e-8278-107f78dc22bd None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:13.926006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e894b86-8f86-4e9e-8278-107f78dc22bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:13.960013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1e894b86-8f86-4e9e-8278-107f78dc22bd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:13.961281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1e894b86-8f86-4e9e-8278-107f78dc22bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:14.098373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 846/1700] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:14.105781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3744bbf-f40c-4dea-8ecb-825bee8e2a2e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:14.106435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3744bbf-f40c-4dea-8ecb-825bee8e2a2e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:14.108914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3744bbf-f40c-4dea-8ecb-825bee8e2a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:14.142580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a3744bbf-f40c-4dea-8ecb-825bee8e2a2e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:14.145710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a3744bbf-f40c-4dea-8ecb-825bee8e2a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:14.212320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2c602-6b1f-49bd-827f-3e5981fb0867 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:14.212617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2c602-6b1f-49bd-827f-3e5981fb0867 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:14.212904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2c602-6b1f-49bd-827f-3e5981fb0867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:14.216958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 855/1701] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:14.223458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3744bbf-f40c-4dea-8ecb-825bee8e2a2e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:14.242615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3744bbf-f40c-4dea-8ecb-825bee8e2a2e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:14.256114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:14.260285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:14.260285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:14.260285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:14.260285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:14.260285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:14.260650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:14.277512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b6c2c602-6b1f-49bd-827f-3e5981fb0867 None None] MFA Rules not processed for user `6214705ae31e459891f7a215f45a4cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:14.278769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b6c2c602-6b1f-49bd-827f-3e5981fb0867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:14.311151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 856/1702] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:27:14.312298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:14.370038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b4e637-fb55-4162-b60f-2e740d2a9fec admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:14.400110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:14.402946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:14.403307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:14.406770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:14.407477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:14.409800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:14.450171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 847/1703] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:14.453738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc2666-8e16-41a4-bfc7-f70eeb1dd315 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:14.457660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc2666-8e16-41a4-bfc7-f70eeb1dd315 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:14.457660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc2666-8e16-41a4-bfc7-f70eeb1dd315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:14.534755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-82dc2666-8e16-41a4-bfc7-f70eeb1dd315 None None] MFA Rules not processed for user `e757ff94b7474fd99a1c7221be19c64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:14.542457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-82dc2666-8e16-41a4-bfc7-f70eeb1dd315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:14.592054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 857/1704] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:14 2022] POST /identity/v3/projects => generated 399 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:14.593215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:14.598440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:14.762245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 848/1705] 10.222.0.60 () {58 vars in 1015 bytes} [Thu May 26 21:27:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:14.763476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc2666-8e16-41a4-bfc7-f70eeb1dd315 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:14.840278 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc2666-8e16-41a4-bfc7-f70eeb1dd315 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:14.860683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:14.887513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:14.930657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c60d61-d144-481d-9f0a-bb9a101bf9cd admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:14.944949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d639bbb-f406-4222-9a0c-50749c6703ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:14.946536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639bbb-f406-4222-9a0c-50749c6703ca service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:14.948037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639bbb-f406-4222-9a0c-50749c6703ca service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:14.948595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d639bbb-f406-4222-9a0c-50749c6703ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.016449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:15.021005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:15.021005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:15.021005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.022543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:15.032019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:15.157058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639bbb-f406-4222-9a0c-50749c6703ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:15.161335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d639bbb-f406-4222-9a0c-50749c6703ca service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:15.179655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 849/1706] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:14 2022] POST /identity/v3/users => generated 300 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:15.180758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:15.183500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:15.215700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 858/1707] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:15.220816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8f2f5-6a41-4827-bc59-63b38ab829da None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:15.221155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8f2f5-6a41-4827-bc59-63b38ab829da None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:15.221451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8f2f5-6a41-4827-bc59-63b38ab829da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.286913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4cd8f2f5-6a41-4827-bc59-63b38ab829da None None] MFA Rules not processed for user `9b94748ea78342629281f2360a54bee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:15.288261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4cd8f2f5-6a41-4827-bc59-63b38ab829da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:15.331855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d30ac-a51c-4003-9d9d-a1d93fde5ff3 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:15.343672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:15.344802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:15.345607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:15.347152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.365728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:15.368560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:15.380972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 850/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:15.382062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:15.402248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f759bb6-d33c-4e6a-8cfe-a9dec81fd22c service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:15.415822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:15.416546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:15.416832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:15.417108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.419946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:15.423348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:15.468946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 851/1709] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:15.475022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 859/1710] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:15.483161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:15.504558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cc965-09a9-469c-99ea-4ad483afa1cd admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:15.521539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d646de0-ee8e-4c22-9c19-c111bccd3e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:15.526991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d646de0-ee8e-4c22-9c19-c111bccd3e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:15.527407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d646de0-ee8e-4c22-9c19-c111bccd3e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:15.527690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d646de0-ee8e-4c22-9c19-c111bccd3e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.538616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d646de0-ee8e-4c22-9c19-c111bccd3e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:15.541610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d646de0-ee8e-4c22-9c19-c111bccd3e9d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:15.600858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8f2f5-6a41-4827-bc59-63b38ab829da None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:15.613310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 852/1711] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:15.619775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8f2f5-6a41-4827-bc59-63b38ab829da None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:15.637146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baebf93-33fd-4985-bb28-9165359ca343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:15.658037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebf93-33fd-4985-bb28-9165359ca343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:15.658037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebf93-33fd-4985-bb28-9165359ca343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:15.658037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baebf93-33fd-4985-bb28-9165359ca343 admin admin] PATH_INFO: `/v3/projects/430f0858b7bc43a8bdacd7360192bd01/users/7c24e22a9ec04b818f210c7aa97fe69d/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.727477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baebf93-33fd-4985-bb28-9165359ca343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=430f0858b7bc43a8bdacd7360192bd01, user_id=7c24e22a9ec04b818f210c7aa97fe69d, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:15.733683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baebf93-33fd-4985-bb28-9165359ca343 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:15.877510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 860/1712] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:15 2022] PUT /identity/v3/projects/430f0858b7bc43a8bdacd7360192bd01/users/7c24e22a9ec04b818f210c7aa97fe69d/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:27:15.882967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea970c8-07a8-49d3-b301-ae09789faae0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:15.883251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea970c8-07a8-49d3-b301-ae09789faae0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:15.883528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea970c8-07a8-49d3-b301-ae09789faae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:15.968094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-eea970c8-07a8-49d3-b301-ae09789faae0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:15.973691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-eea970c8-07a8-49d3-b301-ae09789faae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:16.110219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1719db-f302-4cb6-9b95-f25e29b1a9d3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:16.110585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1719db-f302-4cb6-9b95-f25e29b1a9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:16.111027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1719db-f302-4cb6-9b95-f25e29b1a9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:16.233170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1e1719db-f302-4cb6-9b95-f25e29b1a9d3 None None] MFA Rules not processed for user `ee4307a7bdf84fe18efd6ba3dbe50c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:16.237665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1e1719db-f302-4cb6-9b95-f25e29b1a9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:16.303921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 853/1713] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:16.375065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea970c8-07a8-49d3-b301-ae09789faae0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:16.472241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:16.477603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:16.494629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 861/1714] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:16.515082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:16.515373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:16.515656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:16.584902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] MFA Rules not processed for user `ee4307a7bdf84fe18efd6ba3dbe50c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:16.586432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:16.599848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea970c8-07a8-49d3-b301-ae09789faae0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:16.620495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2817a-a148-47db-8f5b-c5b0673a363e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:16.621180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2817a-a148-47db-8f5b-c5b0673a363e service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:16.621469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2817a-a148-47db-8f5b-c5b0673a363e service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:16.624844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2817a-a148-47db-8f5b-c5b0673a363e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:16.631247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 862/1715] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:16.641581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:16.644683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:16.665809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2817a-a148-47db-8f5b-c5b0673a363e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:16.672995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2817a-a148-47db-8f5b-c5b0673a363e service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:16.741371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 854/1716] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:17.045260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe94667-fc2b-46f6-ad41-ef1d5a3851b3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:17.069472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fce0c3-b536-4d0b-9b8c-395f5980bc1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:17.071657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fce0c3-b536-4d0b-9b8c-395f5980bc1c service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:17.071657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fce0c3-b536-4d0b-9b8c-395f5980bc1c service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:17.071984 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fce0c3-b536-4d0b-9b8c-395f5980bc1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:17.117645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fce0c3-b536-4d0b-9b8c-395f5980bc1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:17.119334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fce0c3-b536-4d0b-9b8c-395f5980bc1c service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:17.142826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 863/1717] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:18.981813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65b19-e9bc-4926-8804-b3e4fdef2f25 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:18.988339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65b19-e9bc-4926-8804-b3e4fdef2f25 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:18.988961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f65b19-e9bc-4926-8804-b3e4fdef2f25 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:18.991260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 855/1718] 10.222.0.42 () {54 vars in 912 bytes} [Thu May 26 21:27:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:27:19.010727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:19.011638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:19.012099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:19.045333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] MFA Rules not processed for user `e757ff94b7474fd99a1c7221be19c64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:19.047447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:19.075397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 864/1719] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:27:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:19.100067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f65b19-e9bc-4926-8804-b3e4fdef2f25 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:19.154370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f65b19-e9bc-4926-8804-b3e4fdef2f25 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:19.182046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9ed99-f8bf-42cc-9ec4-edb601b33cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:19.182340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:19.182340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:19.182494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9ed99-f8bf-42cc-9ec4-edb601b33cac service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:19.182494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9ed99-f8bf-42cc-9ec4-edb601b33cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:19.182494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9ed99-f8bf-42cc-9ec4-edb601b33cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:19.213543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d73c4-68ac-4d9c-b6f3-5d6a5428514a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:19.218492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9ed99-f8bf-42cc-9ec4-edb601b33cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:19.224797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df9ed99-f8bf-42cc-9ec4-edb601b33cac service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:19.228186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:19.247301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:19.248815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:19.249990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] PATH_INFO: `/v3/users/5c7ebce951bf40e3b7eaa2693abfdae4` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:19.254020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7ebce951bf40e3b7eaa2693abfdae4)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:19.260352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 856/1720] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:19.260486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:19.637177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7ebce951bf40e3b7eaa2693abfdae4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:19.715661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7ebce951bf40e3b7eaa2693abfdae4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:19.811654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7ebce951bf40e3b7eaa2693abfdae4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:19.812620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7ebce951bf40e3b7eaa2693abfdae4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:19.813542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0e33fb68-9815-4ad6-b221-2f305d075127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7ebce951bf40e3b7eaa2693abfdae4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:19.831838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 865/1721] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:19 2022] DELETE /identity/v3/users/5c7ebce951bf40e3b7eaa2693abfdae4 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:19.836151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:19.836440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:19.836716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:19.910009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:19.911594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:20.026064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab1441-9761-4db9-af6e-74f6aa2d0658 None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:20.026995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab1441-9761-4db9-af6e-74f6aa2d0658 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:20.027651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab1441-9761-4db9-af6e-74f6aa2d0658 None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:20.030351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 866/1722] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:27:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:27:20.170754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 857/1723] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:20.196618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:20.197251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:20.198069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:20.236727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:20.239685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:20.272772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:20.275090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:20.297137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8676c2af-3ddb-44ae-9124-24c731d59df9 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:20.329780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:20.330791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:20.331810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:20.332309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] PATH_INFO: `/v3/users/38f9c3f296de43cea60525957baed325` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:20.333832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f9c3f296de43cea60525957baed325)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:20.342459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:20.431504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:20.526238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 867/1724] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:20.527993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:20.531908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:20.652424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f9c3f296de43cea60525957baed325'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:20.702806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f9c3f296de43cea60525957baed325'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:20.781328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876dd67-8d4d-4379-85a4-1a747bd5c729 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:20.793792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:20.794873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:20.795307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:20.795725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:20.820143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f9c3f296de43cea60525957baed325'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:20.821350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f9c3f296de43cea60525957baed325'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:20.822274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c0487bc5-aeca-493a-bcd5-908782cedf0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f9c3f296de43cea60525957baed325'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:20.830129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:20.832463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:20.860069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 858/1725] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:20 2022] DELETE /identity/v3/users/38f9c3f296de43cea60525957baed325 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:27:20.868991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:20.869282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:20.869746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:20.885965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 868/1726] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:20.914752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:20.920510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:20.992786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:21.071297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 859/1727] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:21.111040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:21.114608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:21.139402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace4340b-92f6-4737-b648-edabc7b98d20 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:21.152680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:21.154800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:21.223646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b01e5-5a0b-42a3-95ba-813c6b9ec8bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:21.224512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b01e5-5a0b-42a3-95ba-813c6b9ec8bb service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:21.225025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b01e5-5a0b-42a3-95ba-813c6b9ec8bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:21.225527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b01e5-5a0b-42a3-95ba-813c6b9ec8bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:21.255220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b01e5-5a0b-42a3-95ba-813c6b9ec8bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:21.257319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b01e5-5a0b-42a3-95ba-813c6b9ec8bb service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:21.300816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 869/1728] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:21.313695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bfe75-01d6-481f-9f7b-5e3c43c6b5da None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:21.325483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:21.327457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:21.328361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:21.328656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:21.335519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:21.374122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:21.383918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:21.398599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 860/1729] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:22.258301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:22.258849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:22.258934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:22.315906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] MFA Rules not processed for user `ee4307a7bdf84fe18efd6ba3dbe50c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:22.317101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:22.580102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 870/1730] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:23.682548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:23.682548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:23.780477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9330aa2-b387-4615-8f59-0d5823fc7a1e service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:23.805110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:23.816153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:23.828459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:23.828459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] PATH_INFO: `/v3/projects/17ce743073c74a6bb4b0b9dbb651a20f` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:23.828459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ce743073c74a6bb4b0b9dbb651a20f)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:23.828459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:24.505781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:24.507303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:24.602721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae00b3-2e73-4e7f-9168-cb8b3893c676 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:24.602721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:24.602721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:24.602721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:24.603655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] PATH_INFO: `/v3/projects/d2d73f3700b549bdbcc6296883cbb5a0` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:24.603655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2d73f3700b549bdbcc6296883cbb5a0)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:24.603655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:24.603655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:25.258960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2d73f3700b549bdbcc6296883cbb5a0', 'request_id': 'req-73d5fad7-139c-4a59-8094-65fe83393db6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.267096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ce743073c74a6bb4b0b9dbb651a20f', 'request_id': 'req-7af43c3d-87a3-4bcb-a185-239fadc21ab9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.268289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2d73f3700b549bdbcc6296883cbb5a0', 'request_id': 'req-73d5fad7-139c-4a59-8094-65fe83393db6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.293385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ce743073c74a6bb4b0b9dbb651a20f', 'request_id': 'req-7af43c3d-87a3-4bcb-a185-239fadc21ab9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.301926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2d73f3700b549bdbcc6296883cbb5a0', 'request_id': 'req-73d5fad7-139c-4a59-8094-65fe83393db6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.309169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ce743073c74a6bb4b0b9dbb651a20f', 'request_id': 'req-7af43c3d-87a3-4bcb-a185-239fadc21ab9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.436786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] The token cache is being invalidate because project d2d73f3700b549bdbcc6296883cbb5a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:25.436786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.436786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73d5fad7-139c-4a59-8094-65fe83393db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.436786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 871/1731] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:24 2022] DELETE /identity/v3/projects/d2d73f3700b549bdbcc6296883cbb5a0 => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:27:25.436786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161f78-9301-4cfe-9777-a6ea303efc11 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:25.437201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161f78-9301-4cfe-9777-a6ea303efc11 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:25.437201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91161f78-9301-4cfe-9777-a6ea303efc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:25.481879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-91161f78-9301-4cfe-9777-a6ea303efc11 None None] MFA Rules not processed for user `6214705ae31e459891f7a215f45a4cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:25.483259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-91161f78-9301-4cfe-9777-a6ea303efc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:25.528716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] The token cache is being invalidate because project 17ce743073c74a6bb4b0b9dbb651a20f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:25.528716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.528716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:25.531411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 861/1732] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:23 2022] DELETE /identity/v3/projects/17ce743073c74a6bb4b0b9dbb651a20f => generated 0 bytes in 1853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:25.532570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:25.539963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:25.586906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:25.593825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 872/1733] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:25.596514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:25.597130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:25.597656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:25.624241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af43c3d-87a3-4bcb-a185-239fadc21ab9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:25.642262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] MFA Rules not processed for user `7c24e22a9ec04b818f210c7aa97fe69d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:25.643775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:25.702948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:25.705295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:25.705915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:25.706203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] PATH_INFO: `/v3/users/b5f008d5429c455e842d3720f1a95b8d` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:25.711109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f008d5429c455e842d3720f1a95b8d)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:25.716545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:25.774293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 873/1734] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:25.775169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:25.776607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:25.793241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f85785-2daa-45aa-a2cc-d1293e8ade3d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:25.799029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:25.801220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:25.801511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:25.801793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] PATH_INFO: `/v3/users/5bb8360f089e4983aea544a141f782e3` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:25.804514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bb8360f089e4983aea544a141f782e3)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:25.810250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:25.993717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f008d5429c455e842d3720f1a95b8d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.005753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f008d5429c455e842d3720f1a95b8d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.063487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:26.072765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bb8360f089e4983aea544a141f782e3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.091487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bb8360f089e4983aea544a141f782e3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.132468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f008d5429c455e842d3720f1a95b8d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.133242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f008d5429c455e842d3720f1a95b8d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.133689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f008d5429c455e842d3720f1a95b8d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.147655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 862/1735] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:25 2022] DELETE /identity/v3/users/b5f008d5429c455e842d3720f1a95b8d => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:26.148962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:26.150545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:26.251887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a3885-d71e-47e7-8a53-024df78de394 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:26.256795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:26.258552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:26.258552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:26.258552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:26.258786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:26.261044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:26.286405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bb8360f089e4983aea544a141f782e3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.287180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bb8360f089e4983aea544a141f782e3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.287812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bb8360f089e4983aea544a141f782e3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:26.303782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 874/1736] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:25 2022] DELETE /identity/v3/users/5bb8360f089e4983aea544a141f782e3 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:26.404636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:26.406312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:26.409167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 863/1737] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:26 2022] POST /identity/v3/projects => generated 405 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:26.586464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7310246-740d-4a93-ab40-f17c9e3c5048 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:26.722260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:26.723967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:26.724422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:26.724843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:26.725764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:26.728134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:26.819094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 875/1738] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:26 2022] POST /identity/v3/users => generated 309 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:26.824161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:26.839180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e655b-b36c-403d-a4a9-8fd06c59f4db admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:26.845056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:26.846057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:26.846528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:26.846951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:26.847913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:26.853909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:26.867721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:26.875772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 864/1739] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:26.882278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a3b14-a8c3-4a9a-b366-88c214775132 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:26.899615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:26.900809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:26.901092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:26.901366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] PATH_INFO: `/v3/projects/68bdccce40de41e498a6a60a5e07e619/users/84a9fd55901e44e2adacdda52c795cd4/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:26.950054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68bdccce40de41e498a6a60a5e07e619, user_id=84a9fd55901e44e2adacdda52c795cd4, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:26.953036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:26.975805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:26.977947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.024694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cb63d-fc29-4c45-b2cf-83691853bbfe admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.043285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 876/1740] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:26 2022] PUT /identity/v3/projects/68bdccce40de41e498a6a60a5e07e619/users/84a9fd55901e44e2adacdda52c795cd4/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:27.044567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:27.048827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.051709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.075871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.075871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.075871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] PATH_INFO: `/v3/projects/0e961999a82e43b282ab8f84ba1b6d76` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.075871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e961999a82e43b282ab8f84ba1b6d76)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.075871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:27.201419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b33d986-806f-4beb-b349-19d0b5f438e5 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.214695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.216687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.216687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.221311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] PATH_INFO: `/v3/projects/1fd5b62c2f594934810880d6dbbd0c15` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.222568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd5b62c2f594934810880d6dbbd0c15)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.225318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:27.285656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e961999a82e43b282ab8f84ba1b6d76', 'request_id': 'req-49e38e5d-6dc7-453c-9a92-e30177771863'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.293386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e961999a82e43b282ab8f84ba1b6d76', 'request_id': 'req-49e38e5d-6dc7-453c-9a92-e30177771863'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.301403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e961999a82e43b282ab8f84ba1b6d76', 'request_id': 'req-49e38e5d-6dc7-453c-9a92-e30177771863'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.387845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] The token cache is being invalidate because project 0e961999a82e43b282ab8f84ba1b6d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:27.388524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.390228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.393743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 865/1741] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:26 2022] DELETE /identity/v3/projects/0e961999a82e43b282ab8f84ba1b6d76 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:27:27.395391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:27.397377 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.403761 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd5b62c2f594934810880d6dbbd0c15', 'request_id': 'req-e90ade60-8101-4f88-b013-744085516f29'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.414255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd5b62c2f594934810880d6dbbd0c15', 'request_id': 'req-e90ade60-8101-4f88-b013-744085516f29'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.429966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd5b62c2f594934810880d6dbbd0c15', 'request_id': 'req-e90ade60-8101-4f88-b013-744085516f29'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.489923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e38e5d-6dc7-453c-9a92-e30177771863 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.539866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] The token cache is being invalidate because project 1fd5b62c2f594934810880d6dbbd0c15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:27.541994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.541994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e90ade60-8101-4f88-b013-744085516f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:27.543096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 877/1742] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:27 2022] DELETE /identity/v3/projects/1fd5b62c2f594934810880d6dbbd0c15 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:27.544657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aebed-0853-44a3-b900-c94a942fc7f4 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.545177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aebed-0853-44a3-b900-c94a942fc7f4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.547497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8aebed-0853-44a3-b900-c94a942fc7f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.573364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae2a44a-3a77-4650-af9e-95194d477df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.574066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2a44a-3a77-4650-af9e-95194d477df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.574353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2a44a-3a77-4650-af9e-95194d477df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.574636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae2a44a-3a77-4650-af9e-95194d477df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.576388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2a44a-3a77-4650-af9e-95194d477df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.577170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3f8aebed-0853-44a3-b900-c94a942fc7f4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:27.578379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae2a44a-3a77-4650-af9e-95194d477df3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:27.580247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3f8aebed-0853-44a3-b900-c94a942fc7f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.591090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 878/1743] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:27.592837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.593228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.593613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.602831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 866/1744] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:27.613701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8dcd-e997-4108-9fcc-cac9d621bb47 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.613701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8dcd-e997-4108-9fcc-cac9d621bb47 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.613701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff8dcd-e997-4108-9fcc-cac9d621bb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.620097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:27.622143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.638145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 879/1745] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:27.640890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:27.643117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.662565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-14ff8dcd-e997-4108-9fcc-cac9d621bb47 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:27.663869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-14ff8dcd-e997-4108-9fcc-cac9d621bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.665508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5e8cd-559d-4f43-847f-b4637622c544 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.676930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 867/1746] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:27.676930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfbbd3-6d87-4674-b103-40f93688b2a9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.676930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfbbd3-6d87-4674-b103-40f93688b2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.676930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfbbd3-6d87-4674-b103-40f93688b2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.678888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.681613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.682075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.682589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] PATH_INFO: `/v3/projects/68bdccce40de41e498a6a60a5e07e619/users/84a9fd55901e44e2adacdda52c795cd4/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.687083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=68bdccce40de41e498a6a60a5e07e619, user_id=84a9fd55901e44e2adacdda52c795cd4, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.692289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:27.715846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-aabfbbd3-6d87-4674-b103-40f93688b2a9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:27.717772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-aabfbbd3-6d87-4674-b103-40f93688b2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:27.723980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 880/1747] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:27 2022] PUT /identity/v3/projects/68bdccce40de41e498a6a60a5e07e619/users/84a9fd55901e44e2adacdda52c795cd4/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:27.725091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:27.821662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 868/1748] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:27.826123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfbbd3-6d87-4674-b103-40f93688b2a9 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:27.826781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b7138-cd81-4174-a0be-06ce3d126ced admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.832212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.835995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.835995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.835995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.835995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:27.835995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.836346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:27.837988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfbbd3-6d87-4674-b103-40f93688b2a9 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.843183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.843885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.844172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.845069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.850249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 881/1749] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:27:27.857105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:27.857851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.861358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:27.869081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:27.873932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad2721-5e1c-4159-8671-5ae7440fe0b2 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.877185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:27.886263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.888493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.889010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.894703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.894703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.894703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:27.898580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 869/1750] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:27:27.902920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:27.926436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec76b5-b2c4-4ae7-a192-ef09488ea5d1 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:27.937869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:27.939217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:27.939521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:27.939830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:27.940577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:27.943171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.031213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 882/1751] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:27 2022] POST /identity/v3/projects => generated 375 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:28.032421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.092070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 870/1752] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:27 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:27:28.094694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.153549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378e10f-aa87-4208-8a45-12a7b1a3a870 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.254953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e433b-758a-4809-8bfc-826bd25a09d4 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.262282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.263558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.264152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.264439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.265532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.267531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.271599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.273316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.273600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.274250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.274745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.276692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.361091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:28.395614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 871/1753] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:28 2022] POST /identity/v3/users => generated 290 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:28.399848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.410703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84182b-ae33-40bf-a81a-9bc8c62ce5e6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.414352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 883/1754] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:28 2022] POST /identity/v3/users => generated 288 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:28.416638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.418831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.419161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.419460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.420251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.422382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.426662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.436467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 872/1755] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:27:28.441460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.446733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192a493-31db-4f22-8282-f2fcbff42af3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.455538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.457166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.457166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.457553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.458198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.459932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.471497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb4fcd8-d0c7-4544-9d09-32c951de11c2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.474917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 884/1756] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:28.480617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.503943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39122b6-53bd-4de4-9a78-39055c27999d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.510729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.511588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024807d7-7816-4783-98bc-5ec1cef77c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.512023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024807d7-7816-4783-98bc-5ec1cef77c00 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.512317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024807d7-7816-4783-98bc-5ec1cef77c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.512653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024807d7-7816-4783-98bc-5ec1cef77c00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.513530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024807d7-7816-4783-98bc-5ec1cef77c00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.515684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.516625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024807d7-7816-4783-98bc-5ec1cef77c00 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.517491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.520405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.528195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.541852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 885/1757] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:28.547241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c2a83-895e-44c6-afe1-e603ffd2456e None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.547713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c2a83-895e-44c6-afe1-e603ffd2456e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.548186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c2a83-895e-44c6-afe1-e603ffd2456e None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.550837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.552932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 886/1758] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:27:28 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:27:28.554402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c2a83-895e-44c6-afe1-e603ffd2456e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.574787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 873/1759] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:28.579940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.589977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c2a83-895e-44c6-afe1-e603ffd2456e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.598451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba34a6f-7780-4d34-bd68-e362bc20029f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.603404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae464648-ca36-4fd0-b10d-6036c9fd0211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.605257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae464648-ca36-4fd0-b10d-6036c9fd0211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.605257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae464648-ca36-4fd0-b10d-6036c9fd0211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.605257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae464648-ca36-4fd0-b10d-6036c9fd0211 admin admin] PATH_INFO: `/v3/projects/1268647f892744cfafe26cce0f82c4a6/users/77924ef064a6494c89bc1ff70133b7f7/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.609490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c44629-8ce9-4e5a-8163-f883bc1ac27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.610524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44629-8ce9-4e5a-8163-f883bc1ac27d admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.610814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44629-8ce9-4e5a-8163-f883bc1ac27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.611244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c44629-8ce9-4e5a-8163-f883bc1ac27d admin admin] PATH_INFO: `/v3/projects/c34e41edad2f43c791e54c679b847f28/users/7762894958f9442db347a14a44213bf9/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.626727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae464648-ca36-4fd0-b10d-6036c9fd0211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1268647f892744cfafe26cce0f82c4a6, user_id=77924ef064a6494c89bc1ff70133b7f7, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.630489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae464648-ca36-4fd0-b10d-6036c9fd0211 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.633044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c44629-8ce9-4e5a-8163-f883bc1ac27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34e41edad2f43c791e54c679b847f28, user_id=7762894958f9442db347a14a44213bf9, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.634583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c44629-8ce9-4e5a-8163-f883bc1ac27d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.727807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 874/1761] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:28 2022] PUT /identity/v3/projects/c34e41edad2f43c791e54c679b847f28/users/7762894958f9442db347a14a44213bf9/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:28.728108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 887/1761] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:28 2022] PUT /identity/v3/projects/1268647f892744cfafe26cce0f82c4a6/users/77924ef064a6494c89bc1ff70133b7f7/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:28.728601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90579e3-4ae3-4cf3-8620-67b3559cad54 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.728601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90579e3-4ae3-4cf3-8620-67b3559cad54 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.728910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90579e3-4ae3-4cf3-8620-67b3559cad54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.737776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17867a1-165d-4d02-b903-e5b951939e76 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.738066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17867a1-165d-4d02-b903-e5b951939e76 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.738352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17867a1-165d-4d02-b903-e5b951939e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.762339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b90579e3-4ae3-4cf3-8620-67b3559cad54 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:28.764192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b90579e3-4ae3-4cf3-8620-67b3559cad54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:28.773523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f17867a1-165d-4d02-b903-e5b951939e76 None None] MFA Rules not processed for user `77924ef064a6494c89bc1ff70133b7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:28.775010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f17867a1-165d-4d02-b903-e5b951939e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:28.864577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 875/1762] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:28.866289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.866595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.866875 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.881975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 888/1763] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:28.886354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17867a1-165d-4d02-b903-e5b951939e76 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.903645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] MFA Rules not processed for user `7762894958f9442db347a14a44213bf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:28.905751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:28.974985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17867a1-165d-4d02-b903-e5b951939e76 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:28.979838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:28.980591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:28.981339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:28.981339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:28.982038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:28.984163 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:28.993859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 876/1764] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:28.994816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:28.996302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:29.017100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691d2b2e-e253-4543-b4dc-eba75cfd9292 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.023115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.023793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.024625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.024625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.033658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.035816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.046723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 877/1765] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:29.047542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.059398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90492c8a-cd90-42cd-84e3-03f916024209 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.070045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 889/1766] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:28 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:27:29.072442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.073322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.073620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.073885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.074889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.076413 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.079607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.147178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aba01-aadd-4ff0-a5bb-fc56c6063bb4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.151881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.152548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.152835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.153101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.153868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.155808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.223225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 878/1767] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:29 2022] POST /identity/v3/projects => generated 381 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:29.224678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.262177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 890/1768] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:29 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:29.263323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.287162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a6137b-029e-4a54-87d3-7ac508ee7f9d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.313543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d8cec-526d-4702-930c-584a314925f0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.318734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.320379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.320672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.325452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.325452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.325452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.357650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.358224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.358513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.358826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.359731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.361458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.379664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 891/1769] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:27:29.384071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.395423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416569c-2ed6-420c-a39d-2744d1e9db9f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.400740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.406152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.406152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.406152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] PATH_INFO: `/v3/projects/e62e5868c1104901a73aef603c0c6f9a/users/d0f5b7ddb27041f18d210285b863ae35/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.422482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62e5868c1104901a73aef603c0c6f9a, user_id=d0f5b7ddb27041f18d210285b863ae35, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.425266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.437124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 879/1770] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:29 2022] POST /identity/v3/users => generated 297 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:29.441154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.450989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f9b40-71cb-4c1e-a62b-3bb6271bac3f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.455654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.457003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.457336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.457671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.458416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.460064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.470445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 880/1771] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:27:29.475471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.486690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afbea1-1515-4259-81a4-d071b2ba508a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.492762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16e383-debc-4bc4-9e99-1cdb801ec0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.493741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e383-debc-4bc4-9e99-1cdb801ec0f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.499763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e383-debc-4bc4-9e99-1cdb801ec0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.499763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e383-debc-4bc4-9e99-1cdb801ec0f1 admin admin] PATH_INFO: `/v3/projects/53a4b4f08d614c3489356a39e5e794bf/users/aca9ba1e10e34f5f875caf9dca5fe11b/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.509947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16e383-debc-4bc4-9e99-1cdb801ec0f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a4b4f08d614c3489356a39e5e794bf, user_id=aca9ba1e10e34f5f875caf9dca5fe11b, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.512675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16e383-debc-4bc4-9e99-1cdb801ec0f1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.591263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 892/1772] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:29 2022] PUT /identity/v3/projects/e62e5868c1104901a73aef603c0c6f9a/users/d0f5b7ddb27041f18d210285b863ae35/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:29.595195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.664305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 881/1773] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:29 2022] PUT /identity/v3/projects/53a4b4f08d614c3489356a39e5e794bf/users/aca9ba1e10e34f5f875caf9dca5fe11b/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:27:29.667194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947baa3-0800-42f0-9e1d-fb74739b0a48 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.672156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947baa3-0800-42f0-9e1d-fb74739b0a48 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.672156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947baa3-0800-42f0-9e1d-fb74739b0a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.711791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa56d1b-84e4-4558-a364-d0429d0d20e6 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.738076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-a947baa3-0800-42f0-9e1d-fb74739b0a48 None None] MFA Rules not processed for user `84a9fd55901e44e2adacdda52c795cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:29.738266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a947baa3-0800-42f0-9e1d-fb74739b0a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:29.840982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.842183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.842892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.843368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.844915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.847674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.861298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 893/1774] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:27:29.862790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.875326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 882/1775] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:29.876796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947baa3-0800-42f0-9e1d-fb74739b0a48 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.891097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3a885-66b9-4b41-a0d9-582dcd529a46 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.896718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:29.897539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.898273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.898556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.898832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.899617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.901262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.902392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947baa3-0800-42f0-9e1d-fb74739b0a48 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.907184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8924a425-1f04-4c0e-afe3-e4524fd250a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:29.908421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a425-1f04-4c0e-afe3-e4524fd250a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.908799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a425-1f04-4c0e-afe3-e4524fd250a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.909111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924a425-1f04-4c0e-afe3-e4524fd250a4 admin admin] PATH_INFO: `/v3/projects/e62e5868c1104901a73aef603c0c6f9a/users/d0f5b7ddb27041f18d210285b863ae35/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:29.911456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924a425-1f04-4c0e-afe3-e4524fd250a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62e5868c1104901a73aef603c0c6f9a, user_id=d0f5b7ddb27041f18d210285b863ae35, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:29.914522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8924a425-1f04-4c0e-afe3-e4524fd250a4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:29.914794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 894/1776] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:29.919017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:29.933325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 883/1777] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:29 2022] PUT /identity/v3/projects/e62e5868c1104901a73aef603c0c6f9a/users/d0f5b7ddb27041f18d210285b863ae35/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:29.934176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33229e0-bb37-4bee-9de1-0d728ba13cac admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:29.961676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:29.965822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:29.965822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:30.009005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef069cc3-c0a9-4729-b127-1ff2fa94f593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:30.010319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069cc3-c0a9-4729-b127-1ff2fa94f593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:30.010895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069cc3-c0a9-4729-b127-1ff2fa94f593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:30.012176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef069cc3-c0a9-4729-b127-1ff2fa94f593 admin admin] PATH_INFO: `/v3/projects/53a4b4f08d614c3489356a39e5e794bf/users/aca9ba1e10e34f5f875caf9dca5fe11b/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:30.015186 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef069cc3-c0a9-4729-b127-1ff2fa94f593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a4b4f08d614c3489356a39e5e794bf, user_id=aca9ba1e10e34f5f875caf9dca5fe11b, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:30.018916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef069cc3-c0a9-4729-b127-1ff2fa94f593 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:30.045768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 895/1778] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:29 2022] PUT /identity/v3/projects/53a4b4f08d614c3489356a39e5e794bf/users/aca9ba1e10e34f5f875caf9dca5fe11b/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:27:30.075042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0142284-3c02-4085-876b-19341cff5667 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:30.076161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0142284-3c02-4085-876b-19341cff5667 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:30.076544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0142284-3c02-4085-876b-19341cff5667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:30.083340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] MFA Rules not processed for user `d0f5b7ddb27041f18d210285b863ae35`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:30.084588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:30.164453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e0142284-3c02-4085-876b-19341cff5667 None None] MFA Rules not processed for user `aca9ba1e10e34f5f875caf9dca5fe11b`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:30.168501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e0142284-3c02-4085-876b-19341cff5667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:30.238086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 884/1779] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:30.247727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:30.249322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:30.282391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 896/1780] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:30.299056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0142284-3c02-4085-876b-19341cff5667 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:30.301389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e0142284-3c02-4085-876b-19341cff5667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:30.351884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:30.352709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02432d8f-2389-4058-96a3-3a21ae5909d9 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:30.379402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b2cec-093d-4d08-8163-9b716b00e97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:30.383635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b2cec-093d-4d08-8163-9b716b00e97c service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:30.383635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b2cec-093d-4d08-8163-9b716b00e97c service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:30.383884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b2cec-093d-4d08-8163-9b716b00e97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:30.429773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0142284-3c02-4085-876b-19341cff5667 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:30.434601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763efabb-8153-4d37-bc0c-835fa6e3416e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:30.438995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763efabb-8153-4d37-bc0c-835fa6e3416e service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:30.438995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763efabb-8153-4d37-bc0c-835fa6e3416e service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:30.438995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763efabb-8153-4d37-bc0c-835fa6e3416e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:30.446684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763efabb-8153-4d37-bc0c-835fa6e3416e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:30.448325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763efabb-8153-4d37-bc0c-835fa6e3416e service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:30.460585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 897/1781] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:30.521479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b2cec-093d-4d08-8163-9b716b00e97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:30.532102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b2cec-093d-4d08-8163-9b716b00e97c service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:30.569892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 885/1782] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:31.674689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:31.746184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52330ed-5bdb-40c9-828c-a257c3e1b47d None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:31.746734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52330ed-5bdb-40c9-828c-a257c3e1b47d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:31.746734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52330ed-5bdb-40c9-828c-a257c3e1b47d None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:31.748732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 898/1783] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:27:31.769884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:31.770355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:31.770643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:31.790542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52330ed-5bdb-40c9-828c-a257c3e1b47d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:31.800948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:31.802514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:31.804581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52330ed-5bdb-40c9-828c-a257c3e1b47d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:31.811843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aca9ba1e10e34f5f875caf9dca5fe11b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a4b4f08d614c3489356a39e5e794bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:31.812636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:31.813870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:31.814320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:31.816131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:31.816456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:31.818990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:31.862036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 899/1784] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:27:31.862917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:31.886818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef2181-3a49-442c-83f1-484a0d976dc7 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:31.901273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aca9ba1e10e34f5f875caf9dca5fe11b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a4b4f08d614c3489356a39e5e794bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:31.902304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:31.902595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:31.902881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:31.904514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:31.907299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:31.949776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 886/1785] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:27:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:31.976961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:31.979072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:32.006427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ac7925-cec4-4e6d-ab11-587871d67951 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:32.012639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:32.013324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:32.013609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:32.014049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:32.026185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:32.028116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:32.041200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 887/1786] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:32.046746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 900/1787] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:31 2022] POST /identity/v3/projects => generated 354 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:32.056625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:32.076073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744610a1-f9f0-479c-b7bf-d55081092a70 service placement] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:32.093318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:32.172687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aca9ba1e10e34f5f875caf9dca5fe11b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a4b4f08d614c3489356a39e5e794bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:32.173248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:32.173612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:32.173962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:32.174901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:32.175197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:32.177094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:32.197400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 888/1788] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:27:32.243842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:32.257786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658473c6-1bff-4de9-806a-28bc35783b46 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:32.262959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'aca9ba1e10e34f5f875caf9dca5fe11b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '53a4b4f08d614c3489356a39e5e794bf', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:32.264035 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:32.264399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:32.264749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] PATH_INFO: `/v3/projects/264dae3c0afd474db7122c0f782a23b2` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:32.266074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorizing `identity:delete_project(project_id=264dae3c0afd474db7122c0f782a23b2)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:32.268183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:32.401635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264dae3c0afd474db7122c0f782a23b2', 'request_id': 'req-fcc8767f-f85f-4816-9150-223f5fd106f3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:32.422560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264dae3c0afd474db7122c0f782a23b2', 'request_id': 'req-fcc8767f-f85f-4816-9150-223f5fd106f3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:32.433763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264dae3c0afd474db7122c0f782a23b2', 'request_id': 'req-fcc8767f-f85f-4816-9150-223f5fd106f3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:32.556517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] The token cache is being invalidate because project 264dae3c0afd474db7122c0f782a23b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:32.556842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:32.557663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:32.561501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 901/1789] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:32 2022] DELETE /identity/v3/projects/264dae3c0afd474db7122c0f782a23b2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:32.789804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:33.188010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:33.189932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:33.276944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b1be19-d1b9-4d82-a44f-84debe8b171b tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:33.347578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:33.348511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:33.348902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:33.349207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] PATH_INFO: `/v3/users/11ed6e0a382348528286aec0a2c6b2cc` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:33.350435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ed6e0a382348528286aec0a2c6b2cc)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:33.352521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:33.354712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:33.443333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:33.444947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:33.460353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8767f-f85f-4816-9150-223f5fd106f3 tempest-VolumeQuotaClassesTest-1402329011 tempest-VolumeQuotaClassesTest-1402329011-project-admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:33.466173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:33.472258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:33.472258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:33.472258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] PATH_INFO: `/v3/users/7762894958f9442db347a14a44213bf9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:33.472258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7762894958f9442db347a14a44213bf9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:33.472258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:33.583538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ed6e0a382348528286aec0a2c6b2cc'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.599637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ed6e0a382348528286aec0a2c6b2cc'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.740126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ed6e0a382348528286aec0a2c6b2cc'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.741371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ed6e0a382348528286aec0a2c6b2cc'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.742312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7cca3815-8333-43a2-9c86-f5e5f890050a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ed6e0a382348528286aec0a2c6b2cc'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.761067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 889/1790] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:33 2022] DELETE /identity/v3/users/11ed6e0a382348528286aec0a2c6b2cc => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:33.762415 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a249b-013f-4aa3-8383-45ed8c4ea42f None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:33.762701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a249b-013f-4aa3-8383-45ed8c4ea42f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:33.762974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a249b-013f-4aa3-8383-45ed8c4ea42f None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:33.764531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 890/1791] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:27:33.769543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a249b-013f-4aa3-8383-45ed8c4ea42f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:33.769543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-963a249b-013f-4aa3-8383-45ed8c4ea42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:33.803146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7762894958f9442db347a14a44213bf9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.820014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7762894958f9442db347a14a44213bf9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.844676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a249b-013f-4aa3-8383-45ed8c4ea42f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:33.849517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:33.850440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:33.850728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:33.851009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] PATH_INFO: `/v3/users/7c24e22a9ec04b818f210c7aa97fe69d` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:33.852303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c24e22a9ec04b818f210c7aa97fe69d)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:33.854230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:33.892014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7762894958f9442db347a14a44213bf9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.892631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7762894958f9442db347a14a44213bf9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.893258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f14ffc77-6616-4a65-9309-92dfc4e1efdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7762894958f9442db347a14a44213bf9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:33.905695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 902/1792] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:33 2022] DELETE /identity/v3/users/7762894958f9442db347a14a44213bf9 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:33.906951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed47e9-5415-4061-aa14-88a5ce56a76f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:33.907236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed47e9-5415-4061-aa14-88a5ce56a76f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:33.907506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed47e9-5415-4061-aa14-88a5ce56a76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:33.923764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c6ed47e9-5415-4061-aa14-88a5ce56a76f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:33.925590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c6ed47e9-5415-4061-aa14-88a5ce56a76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:33.985448 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 903/1793] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:33.986845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fbb29-3a47-436a-9fab-96bbbfde687e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:33.987800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fbb29-3a47-436a-9fab-96bbbfde687e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:33.988128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072fbb29-3a47-436a-9fab-96bbbfde687e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.007051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-072fbb29-3a47-436a-9fab-96bbbfde687e None None] MFA Rules not processed for user `fd73860db589439da5178e7010d567d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:34.008328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-072fbb29-3a47-436a-9fab-96bbbfde687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.043434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c24e22a9ec04b818f210c7aa97fe69d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:34.060295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c24e22a9ec04b818f210c7aa97fe69d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:34.083055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 904/1794] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:27:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:34.084535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.084908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.088924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.105358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:34.106658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.119902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 905/1795] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:34.120994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:34.125428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.165837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c24e22a9ec04b818f210c7aa97fe69d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:34.167070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c24e22a9ec04b818f210c7aa97fe69d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:34.167716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c24e22a9ec04b818f210c7aa97fe69d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:34.197243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 891/1796] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:33 2022] DELETE /identity/v3/users/7c24e22a9ec04b818f210c7aa97fe69d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:34.198905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:34.209486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.230045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:34.232364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc49708-4467-4040-b148-98d22a401e22 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:34.315049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217ee63-3162-45da-912b-36651e99dd7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:34.315717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217ee63-3162-45da-912b-36651e99dd7d service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.316068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217ee63-3162-45da-912b-36651e99dd7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.316345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217ee63-3162-45da-912b-36651e99dd7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.318191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0217ee63-3162-45da-912b-36651e99dd7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.336028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778c6e19-bff2-4071-8848-6514abe22171 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:34.361540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:34.362705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.362993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.363271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.374223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.402433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:34.404258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:34.408437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217ee63-3162-45da-912b-36651e99dd7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:34.411027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0217ee63-3162-45da-912b-36651e99dd7d service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:34.420847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 892/1797] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:34.422309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:34.422477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 906/1798] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:34.423973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.424179 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9797c3-dd65-443e-be5f-b0c03e92475a None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.425086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9797c3-dd65-443e-be5f-b0c03e92475a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.425376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9797c3-dd65-443e-be5f-b0c03e92475a None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.427090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 907/1799] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:27:34.433986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.434557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.434881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.442761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae753fe8-441c-4ce3-850a-01c54ed18e96 service placement] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:34.453296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:34.453296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.453296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.453296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.453565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.456504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:34.457924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.465141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:34.469532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:34.472142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:34.485152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 893/1800] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:34.561528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 908/1801] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:27:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:34.568464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:34.570988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.594750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a88ef3-b666-4f30-8a7c-5380616498cd service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:34.603910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:34.605341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.606055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.606512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.624604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:34.626457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:34.638733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 894/1802] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:34.765245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:34.871445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:34.873274 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:34.892171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3beef-3313-4203-a9d8-6011b688d03b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:34.903485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:34.903485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:34.903485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:34.903485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] PATH_INFO: `/v3/projects/430f0858b7bc43a8bdacd7360192bd01` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:34.904176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=430f0858b7bc43a8bdacd7360192bd01)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:34.904176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:35.014729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:35.016976 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:35.027718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '430f0858b7bc43a8bdacd7360192bd01', 'request_id': 'req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.033612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a7bc4-35f2-4597-b06f-978c94ad4285 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:35.034710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '430f0858b7bc43a8bdacd7360192bd01', 'request_id': 'req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.042034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '430f0858b7bc43a8bdacd7360192bd01', 'request_id': 'req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.119530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:35.121097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.121581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.122113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] PATH_INFO: `/v3/projects/c34e41edad2f43c791e54c679b847f28` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.124608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c34e41edad2f43c791e54c679b847f28)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:35.134962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:35.137650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] The token cache is being invalidate because project 430f0858b7bc43a8bdacd7360192bd01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:35.138074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.138838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.141069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 909/1803] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:34 2022] DELETE /identity/v3/projects/430f0858b7bc43a8bdacd7360192bd01 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:35.145893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:35.147471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:35.158348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:35.162291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98dda8-dd6b-4e9c-b7ba-401e3c6804e8 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:35.166951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:35.169273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.169273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.169273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] PATH_INFO: `/v3/users/84a9fd55901e44e2adacdda52c795cd4` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.169527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a9fd55901e44e2adacdda52c795cd4)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:35.171471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:35.281363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:35.301813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c34e41edad2f43c791e54c679b847f28', 'request_id': 'req-c8b0754f-53c1-4d29-905b-f5d7553abe4e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.308248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c34e41edad2f43c791e54c679b847f28', 'request_id': 'req-c8b0754f-53c1-4d29-905b-f5d7553abe4e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.314041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c34e41edad2f43c791e54c679b847f28', 'request_id': 'req-c8b0754f-53c1-4d29-905b-f5d7553abe4e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.407322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] The token cache is being invalidate because project c34e41edad2f43c791e54c679b847f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:35.409095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.409462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.411278 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 895/1804] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:35 2022] DELETE /identity/v3/projects/c34e41edad2f43c791e54c679b847f28 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:27:35.414735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:35.416177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:35.464299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a9fd55901e44e2adacdda52c795cd4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.477750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a9fd55901e44e2adacdda52c795cd4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.489014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b0754f-53c1-4d29-905b-f5d7553abe4e admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:35.497027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:35.497905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.498192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.498466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] PATH_INFO: `/v3/projects/9598c083dc2945f698e6c0f561f121bb` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.499807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9598c083dc2945f698e6c0f561f121bb)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:35.504046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:35.520593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a9fd55901e44e2adacdda52c795cd4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.521440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a9fd55901e44e2adacdda52c795cd4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.522108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a9fd55901e44e2adacdda52c795cd4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.537826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 910/1805] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:35 2022] DELETE /identity/v3/users/84a9fd55901e44e2adacdda52c795cd4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:35.538744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:35.540289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:35.619863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a64d1-f159-457d-b230-8808e42f6b35 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:35.636469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9598c083dc2945f698e6c0f561f121bb', 'request_id': 'req-4c50bdee-c863-4985-adf9-d8a68174f967'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.643238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9598c083dc2945f698e6c0f561f121bb', 'request_id': 'req-4c50bdee-c863-4985-adf9-d8a68174f967'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.649481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9598c083dc2945f698e6c0f561f121bb', 'request_id': 'req-4c50bdee-c863-4985-adf9-d8a68174f967'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.684695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:35.685588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.685870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.686146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] PATH_INFO: `/v3/users/aca9ba1e10e34f5f875caf9dca5fe11b` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.687393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca9ba1e10e34f5f875caf9dca5fe11b)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:35.689364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:35.727898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] The token cache is being invalidate because project 9598c083dc2945f698e6c0f561f121bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:35.728195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.728865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4c50bdee-c863-4985-adf9-d8a68174f967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.730466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 896/1806] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:35 2022] DELETE /identity/v3/projects/9598c083dc2945f698e6c0f561f121bb => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:35.735333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1681ae-de4d-4b65-a493-d6057fe519e6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.735618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1681ae-de4d-4b65-a493-d6057fe519e6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.736148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1681ae-de4d-4b65-a493-d6057fe519e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.755188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7a1681ae-de4d-4b65-a493-d6057fe519e6 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:35.759104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7a1681ae-de4d-4b65-a493-d6057fe519e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:35.764420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 897/1807] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:35.772267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cffc1-01b8-4c1c-90c7-c77d9a8626ff None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.772559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cffc1-01b8-4c1c-90c7-c77d9a8626ff None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.772840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7cffc1-01b8-4c1c-90c7-c77d9a8626ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.791477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-4b7cffc1-01b8-4c1c-90c7-c77d9a8626ff None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:35.792720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4b7cffc1-01b8-4c1c-90c7-c77d9a8626ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:35.801326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 898/1808] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:35.806037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cffc1-01b8-4c1c-90c7-c77d9a8626ff None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:35.815171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7cffc1-01b8-4c1c-90c7-c77d9a8626ff None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:35.820223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:35.820923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.821210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.821488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.822609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:35.822924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:35.824703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:35.840592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 899/1809] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:27:35.840592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:35.841381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:35.856662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a26528f-3cfd-42a9-9350-5264b4f4b526 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:35.863301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:35.864236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:35.864537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:35.864807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] PATH_INFO: `/v3/projects/68bdccce40de41e498a6a60a5e07e619` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:35.866031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68bdccce40de41e498a6a60a5e07e619)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:35.868030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:35.891088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca9ba1e10e34f5f875caf9dca5fe11b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.908704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca9ba1e10e34f5f875caf9dca5fe11b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.973985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68bdccce40de41e498a6a60a5e07e619', 'request_id': 'req-00396334-8c01-4f99-ba63-22cdc89af192'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.978365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca9ba1e10e34f5f875caf9dca5fe11b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.979343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca9ba1e10e34f5f875caf9dca5fe11b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.980061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca9ba1e10e34f5f875caf9dca5fe11b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.984221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68bdccce40de41e498a6a60a5e07e619', 'request_id': 'req-00396334-8c01-4f99-ba63-22cdc89af192'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.992953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68bdccce40de41e498a6a60a5e07e619', 'request_id': 'req-00396334-8c01-4f99-ba63-22cdc89af192'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:35.997584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 911/1810] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:35 2022] DELETE /identity/v3/users/aca9ba1e10e34f5f875caf9dca5fe11b => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:36.001539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.003011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.052624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] The token cache is being invalidate because project 68bdccce40de41e498a6a60a5e07e619 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:36.052624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:36.058219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-00396334-8c01-4f99-ba63-22cdc89af192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:36.058219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 900/1811] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:35 2022] DELETE /identity/v3/projects/68bdccce40de41e498a6a60a5e07e619 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:36.062297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6049cb4-7cdc-4513-bd2f-37c6296dd131 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.062721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6049cb4-7cdc-4513-bd2f-37c6296dd131 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.063076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6049cb4-7cdc-4513-bd2f-37c6296dd131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.083862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c6049cb4-7cdc-4513-bd2f-37c6296dd131 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:36.085782 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c6049cb4-7cdc-4513-bd2f-37c6296dd131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.095398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464e051-c9fa-4e17-81c3-ee106000c7c0 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.096966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 901/1812] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:36.106746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5aafc-2737-4d91-9290-d4969b89635b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.106965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5aafc-2737-4d91-9290-d4969b89635b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.107244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc5aafc-2737-4d91-9290-d4969b89635b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.110756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.111440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.111984 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.112308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.113329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.115267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.128271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ccc5aafc-2737-4d91-9290-d4969b89635b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:36.129680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ccc5aafc-2737-4d91-9290-d4969b89635b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.138640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 902/1813] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:36.142713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5aafc-2737-4d91-9290-d4969b89635b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.152756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc5aafc-2737-4d91-9290-d4969b89635b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.157447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.158117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.158398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.158675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.160057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:36.160306 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.161892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.174624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 903/1814] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:27:36.178822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.189727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed0d77e-6928-4a70-b71e-d19f6ec7fa8b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.194618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.195372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.195654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.195961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.196647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.198523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.223060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 912/1815] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:35 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:36.224459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.226064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.285024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 904/1816] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:27:36.288474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.295145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc751f8-2308-40d5-bb7e-4dffdd60d906 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.323229 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:36.368332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:36.370576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.371278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.371567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.371942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.379450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.379601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7edc3b-34c9-4e9c-b9ad-895e6e151040 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.379699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.379810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.380249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.380599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.380911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.381701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.383691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.483822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 913/1817] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:36.483822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.484420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 905/1818] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:36.484507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.485197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.496327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7d70df-49bb-45a5-aafb-d9cb43dd8bf4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.497635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675686a4-3587-4ac2-9ebc-929faa15e9b3 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.501242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.503381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.503734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.504116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] PATH_INFO: `/v3/projects/53a4b4f08d614c3489356a39e5e794bf` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.504666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.505369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.511473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a4b4f08d614c3489356a39e5e794bf)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.511595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.511595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.511595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.511854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.511957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.522767 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 906/1819] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:36.523906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.534566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4335c9-d248-4799-a6ac-501905a0362a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.539361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.540515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.540515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.540916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.541502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.543094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.555108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 907/1820] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:36.556172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.566673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26becc2-5434-4901-8c84-5ee8fea728fa admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.572850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.573528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.573815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.574093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.574827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.576537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.588853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 908/1821] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:36.589185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.599517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f71d5-150a-4b4d-8ead-fb4aba5e3254 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.604620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.605301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.605591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.605869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.606603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.608241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.622920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 909/1822] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:36.622920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.633196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df17c4b-1589-41f1-98c5-b7b97df8291e admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.639244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.640194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.640478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.640754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] PATH_INFO: `/v3/projects/1baf236cb2314f9e9ba0cfbaafa1f961/users/f9af937105114b3bb02a228f5d851a9d/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.655695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a4b4f08d614c3489356a39e5e794bf', 'request_id': 'req-1140daee-a375-4e32-8c47-5231daa6006a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:36.659093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1baf236cb2314f9e9ba0cfbaafa1f961, user_id=f9af937105114b3bb02a228f5d851a9d, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.661966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.667032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a4b4f08d614c3489356a39e5e794bf', 'request_id': 'req-1140daee-a375-4e32-8c47-5231daa6006a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:36.669356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a4b4f08d614c3489356a39e5e794bf', 'request_id': 'req-1140daee-a375-4e32-8c47-5231daa6006a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:36.744715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] The token cache is being invalidate because project 53a4b4f08d614c3489356a39e5e794bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:36.745023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:36.745675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-1140daee-a375-4e32-8c47-5231daa6006a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:36.746497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 910/1823] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:36 2022] PUT /identity/v3/projects/1baf236cb2314f9e9ba0cfbaafa1f961/users/f9af937105114b3bb02a228f5d851a9d/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:36.747292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 914/1824] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:36 2022] DELETE /identity/v3/projects/53a4b4f08d614c3489356a39e5e794bf => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:36.749370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.751059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.752339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ec0eb-ca9a-4735-ac3c-090bd26a19ad None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.753314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ec0eb-ca9a-4735-ac3c-090bd26a19ad None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.753702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ec0eb-ca9a-4735-ac3c-090bd26a19ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.771337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-338ec0eb-ca9a-4735-ac3c-090bd26a19ad None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:36.772733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-338ec0eb-ca9a-4735-ac3c-090bd26a19ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.816907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a9962-2aef-4a5f-b0ff-31eeac158390 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:36.821793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686b2a8-61b8-4388-ab81-2201a41e9d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:36.822769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686b2a8-61b8-4388-ab81-2201a41e9d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.823049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686b2a8-61b8-4388-ab81-2201a41e9d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.823319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686b2a8-61b8-4388-ab81-2201a41e9d9c admin admin] PATH_INFO: `/v3/projects/916d292ee41246c5b4084bf0a74f6eb6/users/0ec2ad9b67f1463fb17592644d298f95/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.832045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 915/1825] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:27:36.833377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cbc2b-7493-4059-93d0-6459c5128cd7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.833713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cbc2b-7493-4059-93d0-6459c5128cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.833964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780cbc2b-7493-4059-93d0-6459c5128cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.839116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686b2a8-61b8-4388-ab81-2201a41e9d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=916d292ee41246c5b4084bf0a74f6eb6, user_id=0ec2ad9b67f1463fb17592644d298f95, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:36.841792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686b2a8-61b8-4388-ab81-2201a41e9d9c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:36.865353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-780cbc2b-7493-4059-93d0-6459c5128cd7 None None] MFA Rules not processed for user `f9af937105114b3bb02a228f5d851a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:36.866542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-780cbc2b-7493-4059-93d0-6459c5128cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.892871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 911/1826] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:36 2022] PUT /identity/v3/projects/916d292ee41246c5b4084bf0a74f6eb6/users/0ec2ad9b67f1463fb17592644d298f95/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:36.894172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.894460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.894740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.913449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:36.914756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.939538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 916/1827] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:36.941280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f64d-197b-4b9f-b9bc-9655fe8bc099 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:36.941567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f64d-197b-4b9f-b9bc-9655fe8bc099 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:36.941856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056f64d-197b-4b9f-b9bc-9655fe8bc099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:36.969547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4056f64d-197b-4b9f-b9bc-9655fe8bc099 None None] MFA Rules not processed for user `0ec2ad9b67f1463fb17592644d298f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:36.970775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4056f64d-197b-4b9f-b9bc-9655fe8bc099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:36.990090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 912/1828] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:36.990936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:36.992273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:37.004976 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e00ec5-4ddd-4915-af9e-e289c30b0a69 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.009746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.010429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.010713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.010983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.011670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.013584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.042069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 917/1829] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:37.043075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f64d-197b-4b9f-b9bc-9655fe8bc099 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.055392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056f64d-197b-4b9f-b9bc-9655fe8bc099 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.060279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.060876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.061107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.061382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.062116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:37.062428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.064051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.076643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 918/1830] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:27:37.080220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.087199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39847cb-fb6f-4f53-b40f-dcd40072a98f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.092301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.092951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.093232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.093510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.094242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.096222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.111276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 913/1831] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:36 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:37.112675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.190469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5382c9-49f1-48f7-bbaf-35ad4b6227b0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.195279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.196015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.196300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.196570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.197279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.199475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.251736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 919/1832] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:37 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:37.251736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.252690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:37.323789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cdfece-7f22-4d54-bdfb-43080b83bf6b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.328867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.334508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.334508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.334508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.334508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.334508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.368164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 914/1833] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:37 2022] POST /identity/v3/projects => generated 409 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:37.369045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.390029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:37.434138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85909104-22f5-47e1-9dd2-9d05f3ba2d37 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.436272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 920/1834] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:37 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:37.437102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.441148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.441796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.442080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.442361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.443062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.445048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.448997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94266926-53ef-479d-a860-a7363b23e2ce admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.453729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.455897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.455897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.455897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.455897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.457550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.547237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 915/1835] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:37 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:37.547546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 921/1836] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:37 2022] POST /identity/v3/users => generated 305 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:37.548081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.549107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.558954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d77a3-84d7-49b2-9695-512305371db2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.560151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462a1bc-1537-40c5-ad8d-683fa9f0067c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.564831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.564831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.564831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.565143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.565236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.565710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.565835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.565995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.566269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.566989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.567379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.568700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.581760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 916/1837] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:37.581760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.582000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 922/1838] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:27:37.583501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.594105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52620692-1f34-4a4a-8068-12eb9a8e2732 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.594280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9aaab-a8b2-4859-b357-e1110bcadb69 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.599473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.599826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.600189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.600474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.600723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.600833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.601012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.601292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] PATH_INFO: `/v3/projects/0413d26c493a46d6910ce20b4b63e251/users/bfd91082e3f14f6182592d14a027284c/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.601466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.603167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.614175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 917/1839] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:37.615138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.620743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0413d26c493a46d6910ce20b4b63e251, user_id=bfd91082e3f14f6182592d14a027284c, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.623415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.627646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bcb7f3-f906-45d0-8dea-e3844ff86bbb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.632560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.633708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.633804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.634107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] PATH_INFO: `/v3/projects/3d0396c65d5f41d28eaa17face6646c3/users/30890de9f3d54f829466453a2086fbed/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.650903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0396c65d5f41d28eaa17face6646c3, user_id=30890de9f3d54f829466453a2086fbed, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.653596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.715645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 923/1840] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:37 2022] PUT /identity/v3/projects/0413d26c493a46d6910ce20b4b63e251/users/bfd91082e3f14f6182592d14a027284c/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:37.719210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.752900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 918/1841] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:37 2022] PUT /identity/v3/projects/3d0396c65d5f41d28eaa17face6646c3/users/30890de9f3d54f829466453a2086fbed/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:37.753862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.785291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a956d-56de-4902-ad81-b20936cf8e54 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.828938 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:37.831087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df52ec6-27db-4148-81b3-57514336fb35 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.836138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.836820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.837109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.837417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.838152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.840179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.851021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 919/1842] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:37.851909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.860801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.861619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.862058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9647c1-9697-43cb-9ef4-1dc929c40d13 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.862334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.862686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.863516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.865595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.873185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.873771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.874054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.874329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.875060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.876513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 924/1843] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:37.878027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.880836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.891561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 920/1844] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:37.891561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.901089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d233d-6a16-4d15-9998-870784cd78c0 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.903558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542638f-d28b-46d2-8858-4641e5207adf admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:37.906120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.907597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.907597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.907904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] PATH_INFO: `/v3/projects/0413d26c493a46d6910ce20b4b63e251/users/bfd91082e3f14f6182592d14a027284c/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.908843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12b3b7-c8ab-4f4a-ab75-27584adc9e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:37.909882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12b3b7-c8ab-4f4a-ab75-27584adc9e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:37.910175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12b3b7-c8ab-4f4a-ab75-27584adc9e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:37.910450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12b3b7-c8ab-4f4a-ab75-27584adc9e40 admin admin] PATH_INFO: `/v3/projects/5a784a65f3094247972ffbcc32167198/users/deb4ed9d2643448182e9942328334022/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:37.910731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0413d26c493a46d6910ce20b4b63e251, user_id=bfd91082e3f14f6182592d14a027284c, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.914302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.926114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12b3b7-c8ab-4f4a-ab75-27584adc9e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a784a65f3094247972ffbcc32167198, user_id=deb4ed9d2643448182e9942328334022, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:37.926419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 925/1845] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:37 2022] PUT /identity/v3/projects/0413d26c493a46d6910ce20b4b63e251/users/bfd91082e3f14f6182592d14a027284c/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:37.927538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:37.934217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12b3b7-c8ab-4f4a-ab75-27584adc9e40 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:37.998909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce008f7-bf00-458b-bd12-0070dfc92d31 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.006384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 921/1846] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:37 2022] PUT /identity/v3/projects/5a784a65f3094247972ffbcc32167198/users/deb4ed9d2643448182e9942328334022/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:38.015185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8ad8-d388-48a1-9e19-bcdfbaf2cdbb None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.015185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8ad8-d388-48a1-9e19-bcdfbaf2cdbb None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.015185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036b8ad8-d388-48a1-9e19-bcdfbaf2cdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.037168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-036b8ad8-d388-48a1-9e19-bcdfbaf2cdbb None None] MFA Rules not processed for user `bfd91082e3f14f6182592d14a027284c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:38.038459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-036b8ad8-d388-48a1-9e19-bcdfbaf2cdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:38.059522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d52f1f-4436-4a36-869d-963ce6794f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.060459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52f1f-4436-4a36-869d-963ce6794f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.060744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52f1f-4436-4a36-869d-963ce6794f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.061020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d52f1f-4436-4a36-869d-963ce6794f43 admin admin] PATH_INFO: `/v3/projects/3d0396c65d5f41d28eaa17face6646c3/users/30890de9f3d54f829466453a2086fbed/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.063061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d52f1f-4436-4a36-869d-963ce6794f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d0396c65d5f41d28eaa17face6646c3, user_id=30890de9f3d54f829466453a2086fbed, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.066330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d52f1f-4436-4a36-869d-963ce6794f43 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.077362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 926/1847] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:37 2022] PUT /identity/v3/projects/3d0396c65d5f41d28eaa17face6646c3/users/30890de9f3d54f829466453a2086fbed/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:38.078670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.079071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.079350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.092768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:38.115603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] MFA Rules not processed for user `deb4ed9d2643448182e9942328334022`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:38.117242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:38.117353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 922/1848] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:38.119171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb3b6-64ed-4890-a75f-dad139126a48 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.119536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb3b6-64ed-4890-a75f-dad139126a48 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.119847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb3b6-64ed-4890-a75f-dad139126a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.154761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-0cbbb3b6-64ed-4890-a75f-dad139126a48 None None] MFA Rules not processed for user `30890de9f3d54f829466453a2086fbed`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:38.156258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0cbbb3b6-64ed-4890-a75f-dad139126a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:38.182729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:38.218550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 927/1849] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:38 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:38.221527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.223658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:38.255229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 923/1850] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:38 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:38.259834 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbb3b6-64ed-4890-a75f-dad139126a48 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.310850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d5a18-652b-4bb5-97de-fc710544ec3d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.324628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.325569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.326181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.326659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.346243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbb3b6-64ed-4890-a75f-dad139126a48 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.352369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.353219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.353697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.353787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.354990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.358287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.435012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.436863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.450913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 928/1851] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:38.451864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.453363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:38.464417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 924/1852] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:38 2022] POST /identity/v3/projects => generated 407 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:38.468672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.475436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f107-4af1-47ad-8c49-b3d290b5a274 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.555323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.556126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.556507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.556885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.589753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa01f9-19f7-4ad2-baf7-ee37e5d4ba35 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.605678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.611132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.611132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.611132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.614587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.618550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.649650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.651373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.664161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 929/1853] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:38.718980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 925/1854] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:38 2022] POST /identity/v3/users => generated 310 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:27:38.720518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.732327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7a418-8775-4578-beeb-31fb657a1eec service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.737850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.738527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.738810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.739251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.740259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.741858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.744334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:38.763230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.782909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3ad11-bdea-4a85-bc19-a1088d3bd4e3 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.788382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 930/1855] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:38.790787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.791451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.791733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.792681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.793353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.802162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:38.817619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86491b6a-e948-4a3b-b526-9cc417073f8b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.833548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.836681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.837090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.838176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] PATH_INFO: `/v3/projects/1039399708b64aa0abac642f13023e69/users/71bb933538204f579593571cdfe044e6/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.876634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=1039399708b64aa0abac642f13023e69, user_id=71bb933538204f579593571cdfe044e6, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.880123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.915999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:38.917465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:38.944729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 926/1856] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:38.944729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:38.954143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32aa9c-3873-4800-afe5-88e850f92e90 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:38.968656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 931/1857] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:38 2022] PUT /identity/v3/projects/1039399708b64aa0abac642f13023e69/users/71bb933538204f579593571cdfe044e6/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:38.972338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:38.972338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:38.975168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:38.975565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:38.975778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:39.073678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:39.075118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:39.086160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ecea3-c1d2-477a-9015-42b799c9e9ae admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:39.088163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 927/1858] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:39.091719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d32df-31ad-4526-920a-ee4d3e7e204f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:39.092491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d32df-31ad-4526-920a-ee4d3e7e204f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:39.092850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d32df-31ad-4526-920a-ee4d3e7e204f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:39.093197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d32df-31ad-4526-920a-ee4d3e7e204f admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:39.094086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d32df-31ad-4526-920a-ee4d3e7e204f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:39.095948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d32df-31ad-4526-920a-ee4d3e7e204f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:39.104263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:39.118019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 932/1859] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:39.122442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:39.135618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ad1d9-0683-48e8-8e26-9bef1748d5cd service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:39.141644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:39.142398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:39.147084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:39.147084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] PATH_INFO: `/v3/projects/1039399708b64aa0abac642f13023e69/users/71bb933538204f579593571cdfe044e6/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:39.147084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1039399708b64aa0abac642f13023e69, user_id=71bb933538204f579593571cdfe044e6, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:39.148396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:39.160620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 928/1860] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:39 2022] PUT /identity/v3/projects/1039399708b64aa0abac642f13023e69/users/71bb933538204f579593571cdfe044e6/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:39.170748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fefd64-bf54-4592-a5db-cc2b29ca79ac None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:39.171034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fefd64-bf54-4592-a5db-cc2b29ca79ac None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:39.175794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fefd64-bf54-4592-a5db-cc2b29ca79ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:39.199180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-83fefd64-bf54-4592-a5db-cc2b29ca79ac None None] MFA Rules not processed for user `71bb933538204f579593571cdfe044e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:39.200433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-83fefd64-bf54-4592-a5db-cc2b29ca79ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:39.296013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 933/1861] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:39 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:39.450873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:39.541731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff560c6-2706-412f-bdb4-b245cb816cb0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:39.547060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:39.548084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:39.548553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:39.549169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:39.561348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:39.563249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:39.579003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 929/1862] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:39 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:40.118602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:40.245813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fefd64-bf54-4592-a5db-cc2b29ca79ac None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:40.266435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fefd64-bf54-4592-a5db-cc2b29ca79ac None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:40.270960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab254423-13e0-4721-9675-6842697ec4c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:40.271622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab254423-13e0-4721-9675-6842697ec4c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:40.271937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab254423-13e0-4721-9675-6842697ec4c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:40.272219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab254423-13e0-4721-9675-6842697ec4c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:40.354463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab254423-13e0-4721-9675-6842697ec4c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:40.356046 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab254423-13e0-4721-9675-6842697ec4c4 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:40.381215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 934/1863] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:40 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:40.864639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:41.029685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:41.343312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:41.350873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:41.440607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91cc7f4-0209-4129-9d5e-682736c1cc7f service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:41.445315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:41.446212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:41.446498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:41.446776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] PATH_INFO: `/v3/users/0ec2ad9b67f1463fb17592644d298f95` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:41.448552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ec2ad9b67f1463fb17592644d298f95)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:41.449980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:41.707242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ec2ad9b67f1463fb17592644d298f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:41.721449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ec2ad9b67f1463fb17592644d298f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:41.848081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ec2ad9b67f1463fb17592644d298f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:41.849993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ec2ad9b67f1463fb17592644d298f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:41.849993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ec2ad9b67f1463fb17592644d298f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:41.862804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 930/1864] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:41 2022] DELETE /identity/v3/users/0ec2ad9b67f1463fb17592644d298f95 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:41.867265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:41.867817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:41.868309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:41.888655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:41.889963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:41.956317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 935/1865] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:41.966914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:41.968493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:42.026176 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6a924-7e1b-4978-9d56-5d715aff4355 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:42.029880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:42.030536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:42.030815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:42.031089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:42.040424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:42.041839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:42.051752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 931/1866] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:42.103582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:42.365923 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:42.530073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:42.531493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:42.544716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2f04ae-4ddb-4068-a327-fee2d045b9b1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:42.558102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:42.558998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:42.566314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:42.566314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] PATH_INFO: `/v3/projects/916d292ee41246c5b4084bf0a74f6eb6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:42.566314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=916d292ee41246c5b4084bf0a74f6eb6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:42.566314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:42.728562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '916d292ee41246c5b4084bf0a74f6eb6', 'request_id': 'req-2ea51718-9bc8-4269-8d21-ce07e967c362'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:42.734834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '916d292ee41246c5b4084bf0a74f6eb6', 'request_id': 'req-2ea51718-9bc8-4269-8d21-ce07e967c362'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:42.742049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '916d292ee41246c5b4084bf0a74f6eb6', 'request_id': 'req-2ea51718-9bc8-4269-8d21-ce07e967c362'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:42.799871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] The token cache is being invalidate because project 916d292ee41246c5b4084bf0a74f6eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:42.800101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:42.800989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:42.803043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 936/1867] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:42 2022] DELETE /identity/v3/projects/916d292ee41246c5b4084bf0a74f6eb6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:42.807327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:42.817707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c0976-4cd1-48f6-8a88-b2420a154e35 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:42.878983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:42.879912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:42.880195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:42.880472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] PATH_INFO: `/v3/users/30890de9f3d54f829466453a2086fbed` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:42.881704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=30890de9f3d54f829466453a2086fbed)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:42.883788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:43.006073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:43.071179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30890de9f3d54f829466453a2086fbed'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.083608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30890de9f3d54f829466453a2086fbed'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.160343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30890de9f3d54f829466453a2086fbed'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.161103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30890de9f3d54f829466453a2086fbed'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.161764 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-77429684-6ecb-4f36-86b2-19ba1c93cd8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30890de9f3d54f829466453a2086fbed'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.174608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 932/1868] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:27:42 2022] DELETE /identity/v3/users/30890de9f3d54f829466453a2086fbed => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:43.229369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:43.422796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:43.489629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:43.646332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:43.648071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:43.665735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea51718-9bc8-4269-8d21-ce07e967c362 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:43.736359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:43.737898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:43.738179 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:43.738456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] PATH_INFO: `/v3/projects/3d0396c65d5f41d28eaa17face6646c3` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:43.739734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d0396c65d5f41d28eaa17face6646c3)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:43.742099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:43.746844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:43.848030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d0396c65d5f41d28eaa17face6646c3', 'request_id': 'req-3b8edb02-861e-48d4-9a71-7401f16c06db'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.854309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d0396c65d5f41d28eaa17face6646c3', 'request_id': 'req-3b8edb02-861e-48d4-9a71-7401f16c06db'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.860021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d0396c65d5f41d28eaa17face6646c3', 'request_id': 'req-3b8edb02-861e-48d4-9a71-7401f16c06db'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.965583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] The token cache is being invalidate because project 3d0396c65d5f41d28eaa17face6646c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:27:43.965876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.966738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3b8edb02-861e-48d4-9a71-7401f16c06db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:27:43.968588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 937/1869] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:27:43 2022] DELETE /identity/v3/projects/3d0396c65d5f41d28eaa17face6646c3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:43.973856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d0be-de4e-464c-bca6-cd5de8071a41 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:43.974178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d0be-de4e-464c-bca6-cd5de8071a41 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:43.974457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de21d0be-de4e-464c-bca6-cd5de8071a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:43.993398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-de21d0be-de4e-464c-bca6-cd5de8071a41 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:43.994635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-de21d0be-de4e-464c-bca6-cd5de8071a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:44.057436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 933/1870] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:44.066498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0ea3-27d9-40fc-a0d4-25054b93c4d9 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.066789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0ea3-27d9-40fc-a0d4-25054b93c4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.067066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed0ea3-27d9-40fc-a0d4-25054b93c4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.085606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-51ed0ea3-27d9-40fc-a0d4-25054b93c4d9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:44.086822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-51ed0ea3-27d9-40fc-a0d4-25054b93c4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:44.094857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 938/1871] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:44.099839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21d0be-de4e-464c-bca6-cd5de8071a41 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.110083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de21d0be-de4e-464c-bca6-cd5de8071a41 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:44.115468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:44.116192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.116502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.116779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.117517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:27:44.117817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:44.119431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:44.132228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 934/1872] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:27:44.136772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0ea3-27d9-40fc-a0d4-25054b93c4d9 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.146724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed0ea3-27d9-40fc-a0d4-25054b93c4d9 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:44.151435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:44.152122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.152409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.152683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.153449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:44.155372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:44.236817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 939/1873] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:44 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:44.241587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.252187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e78f7-b520-4f31-919b-4dc43915feda admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:44.303675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:44.310056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:44.310732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.311048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.311323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.312050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:44.316203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:44.410591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 935/1874] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:44 2022] POST /identity/v3/users => generated 283 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:27:44.414728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.425042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f5c5f-4883-40d2-b74c-66bb01520670 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:44.430612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:44.431287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.431575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.431895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.432704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:44.434428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:44.445929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 940/1875] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:44.450062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.460692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f2e829-2f4b-40a3-8d4e-d08eeff658d8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:44.465835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f13c3-6a6e-4ce2-9750-a0e03ba790ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:44.466495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f13c3-6a6e-4ce2-9750-a0e03ba790ad admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.466773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f13c3-6a6e-4ce2-9750-a0e03ba790ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.467073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f13c3-6a6e-4ce2-9750-a0e03ba790ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.467894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f13c3-6a6e-4ce2-9750-a0e03ba790ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:44.469479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f13c3-6a6e-4ce2-9750-a0e03ba790ad admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:44.480803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 936/1876] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:44.484977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.495041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794b922-1204-4f42-9a6d-e04f564ec970 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:44.499960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:44.500877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.501159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.501438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] PATH_INFO: `/v3/projects/f65b0054809a4796995fa183a10caf76/users/85b00c43127143cc8f8d7af9ab94d7b3/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.518509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65b0054809a4796995fa183a10caf76, user_id=85b00c43127143cc8f8d7af9ab94d7b3, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:44.521282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:44.613338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 941/1877] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:44 2022] PUT /identity/v3/projects/f65b0054809a4796995fa183a10caf76/users/85b00c43127143cc8f8d7af9ab94d7b3/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:44.624895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539c5f-8275-4c94-b00f-de6d774908f6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.625195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539c5f-8275-4c94-b00f-de6d774908f6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.628499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68539c5f-8275-4c94-b00f-de6d774908f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.658653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-68539c5f-8275-4c94-b00f-de6d774908f6 None None] MFA Rules not processed for user `85b00c43127143cc8f8d7af9ab94d7b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:44.664819 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-68539c5f-8275-4c94-b00f-de6d774908f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:44.778869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 937/1878] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:44 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:44.787233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.867012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcaa69-5b7f-4384-8e1d-354533ee3945 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:44.873287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:44.874165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:44.874596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:44.875033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:44.875965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:44.878723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:44.963590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 942/1879] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:27:44 2022] POST /identity/v3/projects => generated 363 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:44.967605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68539c5f-8275-4c94-b00f-de6d774908f6 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:44.982725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68539c5f-8275-4c94-b00f-de6d774908f6 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:45.056675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:45.057376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:45.057662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:45.057940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:45.058672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:45.061204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:45.154430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 938/1880] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:27:44 2022] POST /identity/v3/users => generated 288 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:27:45.159160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:45.164942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:45.170637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efdf1c-d002-485e-b07b-93750beaa0b0 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:45.176064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:45.176756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:45.177041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:45.177321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:45.178059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:45.179990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:45.191687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 943/1881] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:45.196704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:45.206921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0691bd2-59de-4e9e-9fd1-ce6b71bfae89 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:45.211819 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:45.212722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:45.213007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:45.213298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] PATH_INFO: `/v3/projects/e98cd4d29c534041a460c35659773621/users/58dcfd501f0d44238e6b2360fe308760/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:45.228993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98cd4d29c534041a460c35659773621, user_id=58dcfd501f0d44238e6b2360fe308760, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:45.231640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:45.293374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 939/1882] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:45 2022] PUT /identity/v3/projects/e98cd4d29c534041a460c35659773621/users/58dcfd501f0d44238e6b2360fe308760/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:45.297387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:45.308470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30a57b7-cab3-4fd7-a798-c6ba73fca9e6 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:45.370586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3376c-f6bc-443f-9f76-7780e7177878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:45.371257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3376c-f6bc-443f-9f76-7780e7177878 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:45.371684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3376c-f6bc-443f-9f76-7780e7177878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:45.372054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3376c-f6bc-443f-9f76-7780e7177878 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:45.372787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3376c-f6bc-443f-9f76-7780e7177878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:45.374702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3376c-f6bc-443f-9f76-7780e7177878 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:45.389022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 944/1883] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:27:45.390303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:45.400940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de00b0-f27b-4851-ab22-bd9d354e206b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:45.405724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:45.406615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:45.406896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:45.407198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] PATH_INFO: `/v3/projects/e98cd4d29c534041a460c35659773621/users/58dcfd501f0d44238e6b2360fe308760/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:45.410022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98cd4d29c534041a460c35659773621, user_id=58dcfd501f0d44238e6b2360fe308760, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:45.412895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:45.423861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 940/1884] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:27:45 2022] PUT /identity/v3/projects/e98cd4d29c534041a460c35659773621/users/58dcfd501f0d44238e6b2360fe308760/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:27:45.436191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:45.436803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:45.436803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:45.467191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] MFA Rules not processed for user `58dcfd501f0d44238e6b2360fe308760`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:27:45.468472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:45.543624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 945/1885] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:27:45 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:27:45.565364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:45.568142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:45.593442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:45.645020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437927f5-5065-4410-8ec9-cc80fe67c245 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:45.649475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:45.650132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:45.650416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:45.650692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:45.661238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:45.662679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:45.674881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 941/1886] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:45 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:27:46.173394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:27:46.174950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:27:46.195573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71e22ca-fcad-4f57-9c12-385b93a09c0a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:27:46.205270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc33be7-6e44-43ee-ac63-8dee7ad83afd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:27:46.207147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc33be7-6e44-43ee-ac63-8dee7ad83afd service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:27:46.207147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc33be7-6e44-43ee-ac63-8dee7ad83afd service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:27:46.207870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc33be7-6e44-43ee-ac63-8dee7ad83afd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:27:46.298863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc33be7-6e44-43ee-ac63-8dee7ad83afd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:27:46.300711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc33be7-6e44-43ee-ac63-8dee7ad83afd service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:27:46.313174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 946/1887] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:27:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:27:55.703755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=18 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:27:56.307067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:11.359366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:11.484235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372e3595-2feb-41d7-834f-0b2e2d1c8470 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:11.490377 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:11.498512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:11.498512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:11.498512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] PATH_INFO: `/v3/users/85b00c43127143cc8f8d7af9ab94d7b3` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:11.498512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b00c43127143cc8f8d7af9ab94d7b3)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:11.498512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:11.718702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b00c43127143cc8f8d7af9ab94d7b3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:11.741569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b00c43127143cc8f8d7af9ab94d7b3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:11.806364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b00c43127143cc8f8d7af9ab94d7b3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:11.807647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b00c43127143cc8f8d7af9ab94d7b3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:11.809179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b00c43127143cc8f8d7af9ab94d7b3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:11.825983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 942/1888] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:11 2022] DELETE /identity/v3/users/85b00c43127143cc8f8d7af9ab94d7b3 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:11.830743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:11.831262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:11.831550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:11.853254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:11.854865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:11.950941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 947/1889] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:11.960956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:11.962440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:12.000253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:12.002089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:12.015149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ee89-f369-4e56-bf3f-56741af452ca None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:12.020743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:12.021980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:12.021980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:12.022461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] PATH_INFO: `/v3/users/deb4ed9d2643448182e9942328334022` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:12.023619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb4ed9d2643448182e9942328334022)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:12.025649 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:12.047968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e914f-4d0c-4ce5-8681-1125854400e1 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:12.054847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688b307-8e25-48e1-9246-f2feead4d70a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:12.055788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688b307-8e25-48e1-9246-f2feead4d70a service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:12.056314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688b307-8e25-48e1-9246-f2feead4d70a service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:12.056527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688b307-8e25-48e1-9246-f2feead4d70a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:12.069600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688b307-8e25-48e1-9246-f2feead4d70a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:12.071313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688b307-8e25-48e1-9246-f2feead4d70a service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:12.084157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 943/1890] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:12.281139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb4ed9d2643448182e9942328334022'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:12.297483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb4ed9d2643448182e9942328334022'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:12.388714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb4ed9d2643448182e9942328334022'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:12.389250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb4ed9d2643448182e9942328334022'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:12.389873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb4ed9d2643448182e9942328334022'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:12.404147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 948/1891] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:11 2022] DELETE /identity/v3/users/deb4ed9d2643448182e9942328334022 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:12.408877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:12.409378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:12.409501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:12.427126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:12.428756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:12.453467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:12.519786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 944/1892] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:12.532018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:12.533728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:12.616791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18f4df0-4c4d-4f12-9357-ce8a6b52df7c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:12.620980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:12.627020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:12.631711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:12.631711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:12.638340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:12.639932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:12.652800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 949/1893] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:12.728195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:12.733769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:12.757087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f6e02-deaf-49a5-a3d9-8cf79a475f3a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:12.772619 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:12.773888 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:12.774574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:12.775006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] PATH_INFO: `/v3/projects/f65b0054809a4796995fa183a10caf76` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:12.784638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f65b0054809a4796995fa183a10caf76)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:12.787480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:12.941117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f65b0054809a4796995fa183a10caf76', 'request_id': 'req-f5b33840-1b72-461c-a0b9-168cc91ff3a9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:12.948932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f65b0054809a4796995fa183a10caf76', 'request_id': 'req-f5b33840-1b72-461c-a0b9-168cc91ff3a9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:12.956228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f65b0054809a4796995fa183a10caf76', 'request_id': 'req-f5b33840-1b72-461c-a0b9-168cc91ff3a9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.047590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] The token cache is being invalidate because project f65b0054809a4796995fa183a10caf76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:13.047590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.048127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.050644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 945/1894] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:12 2022] DELETE /identity/v3/projects/f65b0054809a4796995fa183a10caf76 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:13.054571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:13.064486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e73e2c-60fb-4d10-aa87-e08b929c6e85 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:13.077176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:13.110354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:13.113035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:13.129699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:13.130760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:13.131124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:13.131474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] PATH_INFO: `/v3/users/58dcfd501f0d44238e6b2360fe308760` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:13.132857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58dcfd501f0d44238e6b2360fe308760)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:13.134878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:13.219289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b33840-1b72-461c-a0b9-168cc91ff3a9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:13.224797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:13.225669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:13.225952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:13.226230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] PATH_INFO: `/v3/projects/5a784a65f3094247972ffbcc32167198` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:13.227438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a784a65f3094247972ffbcc32167198)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:13.229401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:13.343078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:13.344538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58dcfd501f0d44238e6b2360fe308760'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.358071 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a784a65f3094247972ffbcc32167198', 'request_id': 'req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.360824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58dcfd501f0d44238e6b2360fe308760'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.371156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a784a65f3094247972ffbcc32167198', 'request_id': 'req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.377578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a784a65f3094247972ffbcc32167198', 'request_id': 'req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.451629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58dcfd501f0d44238e6b2360fe308760'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.452411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58dcfd501f0d44238e6b2360fe308760'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.453107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58dcfd501f0d44238e6b2360fe308760'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.468307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 950/1895] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:13 2022] DELETE /identity/v3/users/58dcfd501f0d44238e6b2360fe308760 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:13.548567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] The token cache is being invalidate because project 5a784a65f3094247972ffbcc32167198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:13.549134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.549712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.557212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 946/1896] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:13 2022] DELETE /identity/v3/projects/5a784a65f3094247972ffbcc32167198 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:13.558566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:13.560571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:13.633844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2580471-ea17-48a3-abd5-6d45b791b378 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:13.638507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:13.639387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:13.639668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:13.639973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] PATH_INFO: `/v3/users/71bb933538204f579593571cdfe044e6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:13.641162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=71bb933538204f579593571cdfe044e6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:13.643099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:13.943868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:13.952894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71bb933538204f579593571cdfe044e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.966677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71bb933538204f579593571cdfe044e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:13.984108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:13.985736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:14.051268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9f046-f849-4b7c-93d2-2bf56b10b9e3 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.056060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:14.056925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.057207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.057505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] PATH_INFO: `/v3/projects/e98cd4d29c534041a460c35659773621` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.058700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98cd4d29c534041a460c35659773621)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:14.060624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:14.062268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71bb933538204f579593571cdfe044e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.062996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71bb933538204f579593571cdfe044e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.063686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-707f11e2-6f1d-45b1-ae32-06368a120caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71bb933538204f579593571cdfe044e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.077085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 951/1897] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:13 2022] DELETE /identity/v3/users/71bb933538204f579593571cdfe044e6 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:14.194801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98cd4d29c534041a460c35659773621', 'request_id': 'req-5fa1a28c-d4c3-4b93-a529-151781972688'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.201625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98cd4d29c534041a460c35659773621', 'request_id': 'req-5fa1a28c-d4c3-4b93-a529-151781972688'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.214147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98cd4d29c534041a460c35659773621', 'request_id': 'req-5fa1a28c-d4c3-4b93-a529-151781972688'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.313897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] The token cache is being invalidate because project e98cd4d29c534041a460c35659773621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:14.330292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.330292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5fa1a28c-d4c3-4b93-a529-151781972688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.330292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 947/1898] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:13 2022] DELETE /identity/v3/projects/e98cd4d29c534041a460c35659773621 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:14.330602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d95277-cc22-432c-bd12-8cc35e8eedb7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.330602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d95277-cc22-432c-bd12-8cc35e8eedb7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.330602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d95277-cc22-432c-bd12-8cc35e8eedb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.342407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-32d95277-cc22-432c-bd12-8cc35e8eedb7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:14.344098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-32d95277-cc22-432c-bd12-8cc35e8eedb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:14.413755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 952/1899] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:14.422184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5706da-f70f-47f7-b593-552205ac45c7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.422471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5706da-f70f-47f7-b593-552205ac45c7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.422750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5706da-f70f-47f7-b593-552205ac45c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.448448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-3d5706da-f70f-47f7-b593-552205ac45c7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:14.449404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3d5706da-f70f-47f7-b593-552205ac45c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:14.463665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 948/1900] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:14.465134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d95277-cc22-432c-bd12-8cc35e8eedb7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:14.476887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d95277-cc22-432c-bd12-8cc35e8eedb7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.493212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:14.495770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.496059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.496361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.498234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:14.498572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:14.501701 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:14.515535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 953/1901] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:28:14.519914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5706da-f70f-47f7-b593-552205ac45c7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:14.529448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5706da-f70f-47f7-b593-552205ac45c7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.534815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:14.535628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.535998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.542897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.542897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:14.542897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:14.589430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:14.591241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:14.605080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2374fa-660d-4cc3-af39-2d320ec3d0ad admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.611572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:14.613050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.613338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.613683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] PATH_INFO: `/v3/projects/1039399708b64aa0abac642f13023e69` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.614840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1039399708b64aa0abac642f13023e69)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:14.617384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:14.635361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 949/1902] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:14 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:14.638899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:14.687829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:14.708110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d778d-d908-4f51-a80b-bd6c1631d5a7 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.713218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:14.714772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.714772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.714772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.715236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:14.717238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:14.724219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1039399708b64aa0abac642f13023e69', 'request_id': 'req-e08417e2-2955-46b4-a003-72f2d6bcc9c6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.732360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1039399708b64aa0abac642f13023e69', 'request_id': 'req-e08417e2-2955-46b4-a003-72f2d6bcc9c6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.739306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1039399708b64aa0abac642f13023e69', 'request_id': 'req-e08417e2-2955-46b4-a003-72f2d6bcc9c6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.822296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 950/1903] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:14 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:14.825952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:14.835447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785d020-5c54-41f0-b683-cda8503e048b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.856226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] The token cache is being invalidate because project 1039399708b64aa0abac642f13023e69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:14.856515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.857170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e08417e2-2955-46b4-a003-72f2d6bcc9c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:14.858872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 954/1904] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:14 2022] DELETE /identity/v3/projects/1039399708b64aa0abac642f13023e69 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:14.864297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0500f-e4d1-474d-a3c2-b40f91d74497 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.864624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0500f-e4d1-474d-a3c2-b40f91d74497 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.864905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0500f-e4d1-474d-a3c2-b40f91d74497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.882990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-b7d0500f-e4d1-474d-a3c2-b40f91d74497 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:14.884485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b7d0500f-e4d1-474d-a3c2-b40f91d74497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:14.892746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:14.893456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.893740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.894019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.894811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:14.896498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:14.905042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:14.907895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 951/1905] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:14.912228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:14.914111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:14.929122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652a962-bad2-4467-9179-9a97d9d57122 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.936286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:14.938194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.938763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.939387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.940774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:14.943017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:14.951726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 955/1906] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:14.955640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 952/1907] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:14.959507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f631f5-943f-460e-969b-1c73e944517b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:14.959843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f631f5-943f-460e-969b-1c73e944517b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:14.960164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f631f5-943f-460e-969b-1c73e944517b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:14.965420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:14.984985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-14f631f5-943f-460e-969b-1c73e944517b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:14.986208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-14f631f5-943f-460e-969b-1c73e944517b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:14.992151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b5700-d7c5-4284-8925-27b278561c0e admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:14.995874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 956/1908] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:15.001081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f631f5-943f-460e-969b-1c73e944517b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.003263 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9feb4-d811-433a-b88c-0f5e0cc62b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.004398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9feb4-d811-433a-b88c-0f5e0cc62b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.004861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9feb4-d811-433a-b88c-0f5e0cc62b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.005293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9feb4-d811-433a-b88c-0f5e0cc62b4d admin admin] PATH_INFO: `/v3/projects/bcac2a18991a4f98989dc1fa8086723a/users/bff053d5ceb24b2dae37f8724a11fad4/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.020129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f631f5-943f-460e-969b-1c73e944517b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.024313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9feb4-d811-433a-b88c-0f5e0cc62b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcac2a18991a4f98989dc1fa8086723a, user_id=bff053d5ceb24b2dae37f8724a11fad4, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.027037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.027262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9feb4-d811-433a-b88c-0f5e0cc62b4d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.027725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.028063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.028341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.029110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:15.029420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.032917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.046913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 957/1909] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:28:15.052799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.063434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9debd0d5-4bef-4ff8-b306-58b34a79317b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.068733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.069412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.069833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.070127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.070689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.072844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.121488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 953/1910] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:14 2022] PUT /identity/v3/projects/bcac2a18991a4f98989dc1fa8086723a/users/bff053d5ceb24b2dae37f8724a11fad4/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:15.133983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.134273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.134549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.163796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] MFA Rules not processed for user `bff053d5ceb24b2dae37f8724a11fad4`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:15.165029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:15.184362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 958/1911] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:15 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:15.188629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.232429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 954/1912] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:15.244457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.246028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:15.260273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc7113-3aff-4df4-8b40-e4257be34b4d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.265318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.266008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:15.266144 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.266296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.266576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.267268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.269739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.305131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fb9c3-1c47-41b2-8e0c-c6b4ae453eb6 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.351914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.352492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.352838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.353115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.358249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 959/1913] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:15 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:15.359780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.369841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7e82e-a074-4862-b61e-1268f98eb66d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.374600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.375252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.375607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.375855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.376574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.380424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.388992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 960/1914] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:15.392735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.403829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb513ede-5f39-45de-8d9f-2f5c69cb929c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.408994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.412429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.412429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.412429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.412429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.412853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.416394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.417901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.424293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 961/1915] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:15.428171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 955/1916] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:28:15.428774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.439531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24b5658-739a-4ded-a220-1e5a924fb4c9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.445422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3732f0-b9af-4215-9eff-c9182264a255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.446312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3732f0-b9af-4215-9eff-c9182264a255 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.446597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3732f0-b9af-4215-9eff-c9182264a255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.446868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3732f0-b9af-4215-9eff-c9182264a255 admin admin] PATH_INFO: `/v3/projects/d6f3fd2a40bd4ffd80662f357c6c3e41/users/42ad489d8a654315b8e0baf4b26af26e/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.451441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.463238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ab38a-7095-4288-aafa-5b30278ed72d service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.469775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3732f0-b9af-4215-9eff-c9182264a255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f3fd2a40bd4ffd80662f357c6c3e41, user_id=42ad489d8a654315b8e0baf4b26af26e, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.469775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3732f0-b9af-4215-9eff-c9182264a255 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.470056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.470056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.470056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.470056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] PATH_INFO: `/v3/users/bff053d5ceb24b2dae37f8724a11fad4` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.471103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff053d5ceb24b2dae37f8724a11fad4)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.473241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.571056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 962/1917] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:15 2022] PUT /identity/v3/projects/d6f3fd2a40bd4ffd80662f357c6c3e41/users/42ad489d8a654315b8e0baf4b26af26e/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:15.584868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba855a2e-1858-42ef-a72b-63adfb76b004 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.585161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba855a2e-1858-42ef-a72b-63adfb76b004 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.585437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba855a2e-1858-42ef-a72b-63adfb76b004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.611843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ba855a2e-1858-42ef-a72b-63adfb76b004 None None] MFA Rules not processed for user `42ad489d8a654315b8e0baf4b26af26e`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:15.613018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ba855a2e-1858-42ef-a72b-63adfb76b004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:15.625542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:15.671598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff053d5ceb24b2dae37f8724a11fad4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:15.677504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 963/1918] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:15.682473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba855a2e-1858-42ef-a72b-63adfb76b004 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.688760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff053d5ceb24b2dae37f8724a11fad4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:15.753838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba855a2e-1858-42ef-a72b-63adfb76b004 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.758603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.759265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.759550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.759853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.760560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.762474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.800652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff053d5ceb24b2dae37f8724a11fad4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:15.801412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff053d5ceb24b2dae37f8724a11fad4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:15.802088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-03cc6820-17fa-4780-9ed6-37b08c1f7099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff053d5ceb24b2dae37f8724a11fad4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:15.812201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:15.817684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 956/1919] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:15 2022] DELETE /identity/v3/users/bff053d5ceb24b2dae37f8724a11fad4 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:15.822110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.822400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.822693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.840678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:15.841889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:15.873754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 964/1920] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:15 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:15.876472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.878707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:15.901486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 957/1921] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:15.911837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:15.913402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:15.944797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162bc5c-61e5-46f7-a078-b647d40d62dc admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:15.950099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:15.950812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:15.951097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:15.951381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:15.952107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:15.954061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:15.972744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deae1c2-a1bf-47e8-a2b4-3766b26faaf6 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.020833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.021049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.021360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.021640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.031123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.032589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.043398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 958/1922] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:28:16.047782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 965/1923] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:15 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:16.050395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.063868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f374498-b8d8-4ba9-81f1-05d2585dca26 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.069825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.070497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.070779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.071053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.071984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.073722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.086194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 959/1924] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:16.090721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.102049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01959da4-09d5-445a-b64f-ead8f248c65e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.106949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.107878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.108170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.108469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] PATH_INFO: `/v3/projects/6ba83ed338a6477eb23c978e15731dcb/users/7d663bcc4b8143d49acec5d960dfedae/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.125695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba83ed338a6477eb23c978e15731dcb, user_id=7d663bcc4b8143d49acec5d960dfedae, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.128543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.217400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 966/1925] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:16 2022] PUT /identity/v3/projects/6ba83ed338a6477eb23c978e15731dcb/users/7d663bcc4b8143d49acec5d960dfedae/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:16.221939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.232258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f3f67-e10b-4bc6-8f84-5831c84104b4 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.299659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:16.314560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ad5d5-7f8a-4a6c-9f66-a0dc17cdc6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.322296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ad5d5-7f8a-4a6c-9f66-a0dc17cdc6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.322296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ad5d5-7f8a-4a6c-9f66-a0dc17cdc6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.322296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ad5d5-7f8a-4a6c-9f66-a0dc17cdc6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.322296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ad5d5-7f8a-4a6c-9f66-a0dc17cdc6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.322296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ad5d5-7f8a-4a6c-9f66-a0dc17cdc6b6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.332218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 960/1926] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:16.336368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.347918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf492fd-991a-4423-adcd-885ab2f18253 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.352703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.353604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.353888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.354169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] PATH_INFO: `/v3/projects/6ba83ed338a6477eb23c978e15731dcb/users/7d663bcc4b8143d49acec5d960dfedae/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.356195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ba83ed338a6477eb23c978e15731dcb, user_id=7d663bcc4b8143d49acec5d960dfedae, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.358870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.361234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:16.370837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 967/1927] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:16 2022] PUT /identity/v3/projects/6ba83ed338a6477eb23c978e15731dcb/users/7d663bcc4b8143d49acec5d960dfedae/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:16.381050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.381357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.381822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.409655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] MFA Rules not processed for user `7d663bcc4b8143d49acec5d960dfedae`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:16.410896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:16.482919 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 961/1928] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:16.494553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.496261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:16.502444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.503975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:16.570636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da932b9-eb78-485f-b5c3-dcb0335a756b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.574633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba44b73-bac6-4169-ae8a-f781a3344440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.575384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44b73-bac6-4169-ae8a-f781a3344440 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.575676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44b73-bac6-4169-ae8a-f781a3344440 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.575993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44b73-bac6-4169-ae8a-f781a3344440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.577966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bba44b73-bac6-4169-ae8a-f781a3344440 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:16.578855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f33ef7-9574-4d29-ad75-0d94817000aa None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.597396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.597396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.597396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.597396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] PATH_INFO: `/v3/projects/bcac2a18991a4f98989dc1fa8086723a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.598405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcac2a18991a4f98989dc1fa8086723a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.598729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.659868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba44b73-bac6-4169-ae8a-f781a3344440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.661382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba44b73-bac6-4169-ae8a-f781a3344440 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.671072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 968/1929] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:16.733464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcac2a18991a4f98989dc1fa8086723a', 'request_id': 'req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:16.739636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcac2a18991a4f98989dc1fa8086723a', 'request_id': 'req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:16.745912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcac2a18991a4f98989dc1fa8086723a', 'request_id': 'req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:16.790254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:16.794099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] The token cache is being invalidate because project bcac2a18991a4f98989dc1fa8086723a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:16.794390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:16.795036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82f953bf-bc8d-461d-858e-0e22cf4ff2c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:16.796758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 962/1930] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:16 2022] DELETE /identity/v3/projects/bcac2a18991a4f98989dc1fa8086723a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:16.801651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323391-ca57-4377-8205-3e6b076c153b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.801949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323391-ca57-4377-8205-3e6b076c153b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.802300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf323391-ca57-4377-8205-3e6b076c153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.820781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-cf323391-ca57-4377-8205-3e6b076c153b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:16.822206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cf323391-ca57-4377-8205-3e6b076c153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:16.887425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 969/1931] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:16.895643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c83b0-5ab0-4e9f-9bff-ab3bd12d7431 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.896051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c83b0-5ab0-4e9f-9bff-ab3bd12d7431 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.896421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c83b0-5ab0-4e9f-9bff-ab3bd12d7431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.921639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f00c83b0-5ab0-4e9f-9bff-ab3bd12d7431 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:16.923018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f00c83b0-5ab0-4e9f-9bff-ab3bd12d7431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:16.931583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 963/1932] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:16.935868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323391-ca57-4377-8205-3e6b076c153b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.946856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf323391-ca57-4377-8205-3e6b076c153b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.951611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.952315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.952603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.952883 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.953621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:16.953930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.955538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:16.968647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 970/1933] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:28:16.972946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c83b0-5ab0-4e9f-9bff-ab3bd12d7431 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:16.982508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c83b0-5ab0-4e9f-9bff-ab3bd12d7431 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:16.987576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:16.988283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:16.988561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:16.988835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:16.989535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:16.991422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.081658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 964/1934] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:16 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:17.085888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.097398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c9f120-497c-45bb-bdf9-25f32864a519 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.134023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:17.159792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.160473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.160756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.161030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.161725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:17.163678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.281019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 971/1935] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:17 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:17.284394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.294534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20284062-a0c1-4338-99dc-95ef2e8218ff admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.299338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.300659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.301025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.301367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.302159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:17.304155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.316524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 965/1936] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:17.321373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.332079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2f1ac-cdba-4bfb-b337-0b9a3c063b81 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.336758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac08d07f-3505-489e-84a2-52c95642c230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.337518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d07f-3505-489e-84a2-52c95642c230 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.337805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d07f-3505-489e-84a2-52c95642c230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.338076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac08d07f-3505-489e-84a2-52c95642c230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.338973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d07f-3505-489e-84a2-52c95642c230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:17.340689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac08d07f-3505-489e-84a2-52c95642c230 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.352246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 972/1937] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:17.356933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.367661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4f225-a289-4316-bb6a-4b55accf30d4 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.373450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.374487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.374867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.375302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] PATH_INFO: `/v3/projects/03e6c235adce4d82a690fc37128749b5/users/eed568d697764db898d137f6a336adad/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.403648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e6c235adce4d82a690fc37128749b5, user_id=eed568d697764db898d137f6a336adad, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:17.408823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.494873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 966/1938] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:17 2022] PUT /identity/v3/projects/03e6c235adce4d82a690fc37128749b5/users/eed568d697764db898d137f6a336adad/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:17.508822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2330b8a-a0da-4a9e-93e9-e784dd8094ac None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.509143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2330b8a-a0da-4a9e-93e9-e784dd8094ac None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.509479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2330b8a-a0da-4a9e-93e9-e784dd8094ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.540518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a2330b8a-a0da-4a9e-93e9-e784dd8094ac None None] MFA Rules not processed for user `eed568d697764db898d137f6a336adad`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:17.541752 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a2330b8a-a0da-4a9e-93e9-e784dd8094ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:17.609636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 973/1939] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:17.615148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.698625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ee0cc-59c9-4449-9ec3-41bbdc86713c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.703669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.704389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.704681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.705056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.705691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:17.707610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.796266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 967/1940] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:17 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:17.801157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2330b8a-a0da-4a9e-93e9-e784dd8094ac None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.812655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2330b8a-a0da-4a9e-93e9-e784dd8094ac None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.875348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.876066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.876349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.876635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.877337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:17.880050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.944677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 974/1941] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:17 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:17.949125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.960384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e08cf-46f4-451b-b25f-d419d88fbf4a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.965004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.965674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:17.965943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:17.966216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:17.966928 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:17.968565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:17.979331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 968/1942] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:17.983524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:17.994075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20819308-9d8b-4a66-beda-18eba6e9073e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:17.999191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6217d7c7-1a3e-4618-89ce-cc6e57b60e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:17.999889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d7c7-1a3e-4618-89ce-cc6e57b60e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.000174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d7c7-1a3e-4618-89ce-cc6e57b60e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.000451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6217d7c7-1a3e-4618-89ce-cc6e57b60e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.001220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217d7c7-1a3e-4618-89ce-cc6e57b60e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.003978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6217d7c7-1a3e-4618-89ce-cc6e57b60e1b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.016845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 975/1943] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:18.021815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:18.032184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31bba6-7d99-4586-9740-383ccf8a4158 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:18.036992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:18.037884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.038168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.038482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] PATH_INFO: `/v3/projects/896b3030a4a04f74b9d6568bc3abfba0/users/4c27eb469eb54930a91a58780cc0f371/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.057310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=896b3030a4a04f74b9d6568bc3abfba0, user_id=4c27eb469eb54930a91a58780cc0f371, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.060234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.160623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 969/1944] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:18 2022] PUT /identity/v3/projects/896b3030a4a04f74b9d6568bc3abfba0/users/4c27eb469eb54930a91a58780cc0f371/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:18.171217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37d33-3568-47ea-afb3-63df0c72cf4f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.171643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37d33-3568-47ea-afb3-63df0c72cf4f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.172018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c37d33-3568-47ea-afb3-63df0c72cf4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.220672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d9c37d33-3568-47ea-afb3-63df0c72cf4f None None] MFA Rules not processed for user `4c27eb469eb54930a91a58780cc0f371`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:18.221837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d9c37d33-3568-47ea-afb3-63df0c72cf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:18.236694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:18.296972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 976/1945] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:18.302043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:18.365922 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:18.379831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff17b1-180e-4f16-a3ae-ce5bc52c5c25 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:18.384535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:18.385197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.385485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.385764 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.386464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.388399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.490357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 970/1946] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:18 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:18.493921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37d33-3568-47ea-afb3-63df0c72cf4f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:18.505423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c37d33-3568-47ea-afb3-63df0c72cf4f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:18.569844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:18.570546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.570836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.571119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.571865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.573828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.654945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 977/1947] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:18 2022] POST /identity/v3/users => generated 294 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:18.656333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:18.668203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3edf64-d499-433d-967e-9d6a74d913e1 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:18.674191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:18.675095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.675549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.676626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.677512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.679545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.699631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 971/1948] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:18.704124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:18.717569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c98a3-57d3-4dad-8c75-30da3cbbb68e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:18.726162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:18.727719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.728255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.728760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] PATH_INFO: `/v3/projects/5f702fca9371466386692e14fb5a77f7/users/12b5ab491cc44ede929571b6801ccb4a/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.753591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f702fca9371466386692e14fb5a77f7, user_id=12b5ab491cc44ede929571b6801ccb4a, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.756428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.828719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 978/1949] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:18 2022] PUT /identity/v3/projects/5f702fca9371466386692e14fb5a77f7/users/12b5ab491cc44ede929571b6801ccb4a/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:18.833226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:18.843359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23c2d7-f4be-41c5-9dee-be46f694cf93 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:18.898817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac27d7a-ab9a-4f18-93fc-f30207e1120d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:18.899850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac27d7a-ab9a-4f18-93fc-f30207e1120d admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.900279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac27d7a-ab9a-4f18-93fc-f30207e1120d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.900721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac27d7a-ab9a-4f18-93fc-f30207e1120d admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.901598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac27d7a-ab9a-4f18-93fc-f30207e1120d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.903377 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac27d7a-ab9a-4f18-93fc-f30207e1120d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.916599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 972/1950] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:18.920646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:18.932687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59843723-1ca2-4f95-bf46-6a5f2b81baf4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:18.938062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:18.938953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.939237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.939514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] PATH_INFO: `/v3/projects/5f702fca9371466386692e14fb5a77f7/users/12b5ab491cc44ede929571b6801ccb4a/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:18.941683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f702fca9371466386692e14fb5a77f7, user_id=12b5ab491cc44ede929571b6801ccb4a, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:18.944856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:18.959334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 979/1951] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:18 2022] PUT /identity/v3/projects/5f702fca9371466386692e14fb5a77f7/users/12b5ab491cc44ede929571b6801ccb4a/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:18.970235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf31eb8-ad0a-4c58-a0d7-0f54e0be23da None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:18.970851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf31eb8-ad0a-4c58-a0d7-0f54e0be23da None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:18.971133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf31eb8-ad0a-4c58-a0d7-0f54e0be23da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:19.012765 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ecf31eb8-ad0a-4c58-a0d7-0f54e0be23da None None] MFA Rules not processed for user `12b5ab491cc44ede929571b6801ccb4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:19.013981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ecf31eb8-ad0a-4c58-a0d7-0f54e0be23da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:19.087674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 973/1952] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:19.099637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:19.101801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:19.163205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724e4c8-4590-4f41-a0d3-aab53d29f2af admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:19.166943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:19.167600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:19.167920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:19.168200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:19.188141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:19.228908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:19.230351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:19.239571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 980/1953] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:22.322482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf31eb8-ad0a-4c58-a0d7-0f54e0be23da None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:22.341218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf31eb8-ad0a-4c58-a0d7-0f54e0be23da None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:22.345934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:22.346827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:22.347132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:22.347412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:22.393990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:22.417812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:22.417812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:22.430590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 974/1954] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:24.726112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:24.748009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a246e-f319-443c-81f4-644903b910dd service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:24.751818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:24.752571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:24.752854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:24.753129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:24.788986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:24.790633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:24.800666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 981/1955] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:25.561067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:25.583817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0544b-1b20-4f05-a12d-30265557a613 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:25.588042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d4403-96ab-4ba5-bb81-a07c1cdd2ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:25.588738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4403-96ab-4ba5-bb81-a07c1cdd2ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:25.589014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4403-96ab-4ba5-bb81-a07c1cdd2ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:25.589292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4403-96ab-4ba5-bb81-a07c1cdd2ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:25.591258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2f3d4403-96ab-4ba5-bb81-a07c1cdd2ca1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:25.677273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d4403-96ab-4ba5-bb81-a07c1cdd2ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:25.678986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d4403-96ab-4ba5-bb81-a07c1cdd2ca1 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:25.689996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 975/1956] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:25.733412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:25.735105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:25.823491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430116c5-dc9c-42b0-8395-2b26d49782dc service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:25.829527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:25.830492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:25.838382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:25.838382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] PATH_INFO: `/v3/users/77924ef064a6494c89bc1ff70133b7f7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:25.838382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77924ef064a6494c89bc1ff70133b7f7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:25.838382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:26.152260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77924ef064a6494c89bc1ff70133b7f7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:26.190633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77924ef064a6494c89bc1ff70133b7f7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:26.193518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:26.270893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77924ef064a6494c89bc1ff70133b7f7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:26.272985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77924ef064a6494c89bc1ff70133b7f7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:26.273876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77924ef064a6494c89bc1ff70133b7f7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:26.292396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 982/1957] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:25 2022] DELETE /identity/v3/users/77924ef064a6494c89bc1ff70133b7f7 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:26.296913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:26.297121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:26.297482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:26.315121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:26.317076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:26.378802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 976/1958] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:26.389419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:26.390956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:26.469173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ea1a5-a297-4330-a878-10792175b728 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:26.473131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:26.473886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:26.474242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:26.474585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:26.489473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:26.490338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:26.509595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 983/1959] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:27.184260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:27.189575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:27.198488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61233b79-854c-47b4-8109-6624fd1a80c0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:27.203256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:27.204175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:27.204653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:27.204998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] PATH_INFO: `/v3/projects/1268647f892744cfafe26cce0f82c4a6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:27.206181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1268647f892744cfafe26cce0f82c4a6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:27.208757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:27.382792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1268647f892744cfafe26cce0f82c4a6', 'request_id': 'req-13688a5a-8423-4a5d-aa3d-d47dabe34e63'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.384660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:27.399874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1268647f892744cfafe26cce0f82c4a6', 'request_id': 'req-13688a5a-8423-4a5d-aa3d-d47dabe34e63'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.411077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1268647f892744cfafe26cce0f82c4a6', 'request_id': 'req-13688a5a-8423-4a5d-aa3d-d47dabe34e63'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.525096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] The token cache is being invalidate because project 1268647f892744cfafe26cce0f82c4a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:27.525536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.526303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.530041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 977/1960] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:27 2022] DELETE /identity/v3/projects/1268647f892744cfafe26cce0f82c4a6 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:27.532428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:27.548052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f88c79d-a04a-424d-92a2-c7f257cf3e49 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:27.631561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:27.632573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:27.632921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:27.633261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] PATH_INFO: `/v3/users/d0f5b7ddb27041f18d210285b863ae35` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:27.634577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0f5b7ddb27041f18d210285b863ae35)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:27.636663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:27.862240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0f5b7ddb27041f18d210285b863ae35'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.886882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0f5b7ddb27041f18d210285b863ae35'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.947168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0f5b7ddb27041f18d210285b863ae35'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.947906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0f5b7ddb27041f18d210285b863ae35'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.948537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-e82f4416-d83d-4549-b4cb-48a67f6fd217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0f5b7ddb27041f18d210285b863ae35'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:27.954835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:27.964024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 984/1961] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:27 2022] DELETE /identity/v3/users/d0f5b7ddb27041f18d210285b863ae35 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:28.418610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:28.420958 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:28.443230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13688a5a-8423-4a5d-aa3d-d47dabe34e63 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:28.509067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:28.511792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:28.511792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:28.511792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] PATH_INFO: `/v3/projects/e62e5868c1104901a73aef603c0c6f9a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:28.511792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62e5868c1104901a73aef603c0c6f9a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:28.513776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:28.676927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62e5868c1104901a73aef603c0c6f9a', 'request_id': 'req-602ae1d2-8835-4812-a81b-649672b39c87'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:28.690773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62e5868c1104901a73aef603c0c6f9a', 'request_id': 'req-602ae1d2-8835-4812-a81b-649672b39c87'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:28.706187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62e5868c1104901a73aef603c0c6f9a', 'request_id': 'req-602ae1d2-8835-4812-a81b-649672b39c87'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:28.776621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] The token cache is being invalidate because project e62e5868c1104901a73aef603c0c6f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:28.776621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:28.777719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-602ae1d2-8835-4812-a81b-649672b39c87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:28.779296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 978/1962] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:28 2022] DELETE /identity/v3/projects/e62e5868c1104901a73aef603c0c6f9a => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:28.784852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d0c1-998b-44b7-ae77-c54ca5b2e91f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:28.785458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d0c1-998b-44b7-ae77-c54ca5b2e91f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:28.785887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5d0c1-998b-44b7-ae77-c54ca5b2e91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:28.851018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a5c5d0c1-998b-44b7-ae77-c54ca5b2e91f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:28.852494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a5c5d0c1-998b-44b7-ae77-c54ca5b2e91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:28.952908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 985/1963] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:28.960188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3fc13-52c8-456b-8a21-637557fcf721 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:28.960756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3fc13-52c8-456b-8a21-637557fcf721 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:28.961484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a3fc13-52c8-456b-8a21-637557fcf721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:28.983536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-88a3fc13-52c8-456b-8a21-637557fcf721 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:28.985081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-88a3fc13-52c8-456b-8a21-637557fcf721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:28.996444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 979/1964] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:29.001309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5d0c1-998b-44b7-ae77-c54ca5b2e91f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:29.013743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5d0c1-998b-44b7-ae77-c54ca5b2e91f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:29.019406 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:29.020151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.020460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.020719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.021454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:29.021763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:29.023377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:29.036749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 986/1965] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:28:29.041235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3fc13-52c8-456b-8a21-637557fcf721 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:29.050422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a3fc13-52c8-456b-8a21-637557fcf721 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:29.056446 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:29.057186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.057553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.057829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.058543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:29.060488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:29.179420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 980/1966] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:29 2022] POST /identity/v3/projects => generated 391 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:29.183506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:29.194377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a12df3d-e7c0-4cfa-955b-84ea5f6cee5d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:29.284004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:29.286307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.286307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.286307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.286606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:29.288561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:29.421521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 987/1967] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:29 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:29.422822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:29.433627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e477aef-4fb0-44e3-9714-59372a584b12 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:29.438520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:29.439205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.439481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.439975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.440977 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:29.442292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:29.464386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 981/1968] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:29.468363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:29.481538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528bb227-65b1-47f2-8b46-6647e10ae4da admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:29.489992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0104cba-8e9b-4511-9fa4-1a7389f9cafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:29.491021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0104cba-8e9b-4511-9fa4-1a7389f9cafb admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.491362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0104cba-8e9b-4511-9fa4-1a7389f9cafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.491870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0104cba-8e9b-4511-9fa4-1a7389f9cafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.499027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0104cba-8e9b-4511-9fa4-1a7389f9cafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:29.499027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0104cba-8e9b-4511-9fa4-1a7389f9cafb admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:29.511008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 988/1969] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:29.512258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:29.523658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649fb97-8fa8-4ce2-b30f-e537f4845a71 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:29.539474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:29.540591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.540975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.541321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] PATH_INFO: `/v3/projects/ba4306524f144654bdfe63b6c1394bac/users/e82c194591c24998a7812047df6e16aa/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.558024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba4306524f144654bdfe63b6c1394bac, user_id=e82c194591c24998a7812047df6e16aa, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:29.561044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:29.657624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 982/1970] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:29 2022] PUT /identity/v3/projects/ba4306524f144654bdfe63b6c1394bac/users/e82c194591c24998a7812047df6e16aa/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:29.669045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656b6f-3d97-4028-b0db-770fec9845bd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.669339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656b6f-3d97-4028-b0db-770fec9845bd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.669623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87656b6f-3d97-4028-b0db-770fec9845bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.713756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-87656b6f-3d97-4028-b0db-770fec9845bd None None] MFA Rules not processed for user `e82c194591c24998a7812047df6e16aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:29.716647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-87656b6f-3d97-4028-b0db-770fec9845bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:29.740181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:29.825127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 989/1971] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:29.829862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:29.836795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:29.936177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e5439-1b5b-4503-9708-2e27a7b4eaa6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:29.942156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:29.942909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:29.943294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:29.943572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:29.944327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:29.946259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.067134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 983/1972] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:29 2022] POST /identity/v3/projects => generated 393 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:30.068337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87656b6f-3d97-4028-b0db-770fec9845bd None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:30.079355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87656b6f-3d97-4028-b0db-770fec9845bd None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:30.138941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:30.139784 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.140134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.140417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.143852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:30.148331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.234204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 990/1973] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:30 2022] POST /identity/v3/users => generated 303 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:30.238478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:30.248904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1a999-a554-4264-ac47-c9730e3c9e50 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:30.253570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:30.254170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.254452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.254729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.255451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:30.257284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.268073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 984/1974] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:30.272976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:30.283025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c71d37-8f5e-426f-8071-67407368826e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:30.288303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:30.289185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.289475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.289753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] PATH_INFO: `/v3/projects/fef7fb75ca6744369ecf3883626f6ace/users/813c17bdd3584232b8d08645cc501fd7/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.304754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef7fb75ca6744369ecf3883626f6ace, user_id=813c17bdd3584232b8d08645cc501fd7, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:30.307482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.457922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 991/1975] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:30 2022] PUT /identity/v3/projects/fef7fb75ca6744369ecf3883626f6ace/users/813c17bdd3584232b8d08645cc501fd7/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:30.461921 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:30.472089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57a582-0f88-46a5-8592-2ad28ab36c31 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:30.530260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4b837-17bc-41cf-81ab-37a26c5b560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:30.530851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b837-17bc-41cf-81ab-37a26c5b560c admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.531133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b837-17bc-41cf-81ab-37a26c5b560c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.531434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4b837-17bc-41cf-81ab-37a26c5b560c admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.532194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4b837-17bc-41cf-81ab-37a26c5b560c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:30.533827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4b837-17bc-41cf-81ab-37a26c5b560c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.546014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 985/1976] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:30.554250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:30.561497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9393d63-cb68-4ef5-8edf-4fea3e112b9d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:30.570716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:30.571765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.572154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.572666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] PATH_INFO: `/v3/projects/fef7fb75ca6744369ecf3883626f6ace/users/813c17bdd3584232b8d08645cc501fd7/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.575141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef7fb75ca6744369ecf3883626f6ace, user_id=813c17bdd3584232b8d08645cc501fd7, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:30.578036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.595930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 992/1977] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:30 2022] PUT /identity/v3/projects/fef7fb75ca6744369ecf3883626f6ace/users/813c17bdd3584232b8d08645cc501fd7/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:30.601856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.602253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.602745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.630260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] MFA Rules not processed for user `813c17bdd3584232b8d08645cc501fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:30.631625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:30.713276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 986/1978] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:30 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:30.724346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:30.726032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:30.783876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a202f-e6ff-46be-9524-b570b35a52bb admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:30.789297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:30.793404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:30.794082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.794370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.794650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.807212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:30.808857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.820737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 993/1979] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:30 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:30.875652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:30.877102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:30.889635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f0509d-c6df-4b36-8614-bd282b46c1f0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:30.893453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5aee18-7250-4211-8be9-c44ea8225bf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:30.899716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aee18-7250-4211-8be9-c44ea8225bf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:30.899716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aee18-7250-4211-8be9-c44ea8225bf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:30.899716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5aee18-7250-4211-8be9-c44ea8225bf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:30.963367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5aee18-7250-4211-8be9-c44ea8225bf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:30.964915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5aee18-7250-4211-8be9-c44ea8225bf5 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:30.979951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 987/1980] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:31.295515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:31.296952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:31.307834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:31.362178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28648ea-d4ba-4be1-8109-63124931a3e9 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:31.367472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:31.368373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:31.368655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:31.368925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] PATH_INFO: `/v3/users/eed568d697764db898d137f6a336adad` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:31.370216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed568d697764db898d137f6a336adad)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:31.372210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:31.585891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed568d697764db898d137f6a336adad'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:31.598298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed568d697764db898d137f6a336adad'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:31.656757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed568d697764db898d137f6a336adad'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:31.657539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed568d697764db898d137f6a336adad'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:31.658737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed568d697764db898d137f6a336adad'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:31.672716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 994/1981] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:31 2022] DELETE /identity/v3/users/eed568d697764db898d137f6a336adad => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:31.677312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:31.680621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:31.680621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:31.695429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:31.696900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:31.787382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 988/1982] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:31.805845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:31.808484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:31.944851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f98619-dc2d-4ef0-9660-566d6667a06a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:31.948513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:31.949159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:31.949440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:31.949715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:31.959115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:31.960595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:31.970212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 995/1983] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:32.732969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:32.736576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:32.751531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0637d0-07f8-486e-b254-86a20af70503 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:32.757441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:32.758316 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:32.758594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:32.759173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] PATH_INFO: `/v3/projects/03e6c235adce4d82a690fc37128749b5` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:32.760556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e6c235adce4d82a690fc37128749b5)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:32.762460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:32.914683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e6c235adce4d82a690fc37128749b5', 'request_id': 'req-4030f972-1137-4772-afe9-fcdb78b8e6f9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:32.925021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e6c235adce4d82a690fc37128749b5', 'request_id': 'req-4030f972-1137-4772-afe9-fcdb78b8e6f9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:32.934393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e6c235adce4d82a690fc37128749b5', 'request_id': 'req-4030f972-1137-4772-afe9-fcdb78b8e6f9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.051148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] The token cache is being invalidate because project 03e6c235adce4d82a690fc37128749b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:33.051577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.052164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.054039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 989/1984] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:32 2022] DELETE /identity/v3/projects/03e6c235adce4d82a690fc37128749b5 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:33.058211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:33.068506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f46611-e71f-4be6-87e6-c5a6df5e4874 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:33.073188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:33.125408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:33.126365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:33.126770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:33.126943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] PATH_INFO: `/v3/users/4c27eb469eb54930a91a58780cc0f371` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:33.128354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c27eb469eb54930a91a58780cc0f371)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:33.130367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:33.225066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:33.228157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:33.246090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4030f972-1137-4772-afe9-fcdb78b8e6f9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:33.251160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:33.252165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:33.252448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:33.252823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] PATH_INFO: `/v3/users/42ad489d8a654315b8e0baf4b26af26e` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:33.254388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42ad489d8a654315b8e0baf4b26af26e)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:33.256353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:33.273992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:33.339027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c27eb469eb54930a91a58780cc0f371'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.354626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c27eb469eb54930a91a58780cc0f371'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.423304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c27eb469eb54930a91a58780cc0f371'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.424337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c27eb469eb54930a91a58780cc0f371'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.425161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-00fedbfa-ed9c-414c-a1ad-2fc0e311b984 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c27eb469eb54930a91a58780cc0f371'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.439712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 996/1985] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:33 2022] DELETE /identity/v3/users/4c27eb469eb54930a91a58780cc0f371 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:33.440084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42ad489d8a654315b8e0baf4b26af26e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.469188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42ad489d8a654315b8e0baf4b26af26e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.525113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42ad489d8a654315b8e0baf4b26af26e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.525794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42ad489d8a654315b8e0baf4b26af26e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.526450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42ad489d8a654315b8e0baf4b26af26e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:33.552766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 990/1986] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:33 2022] DELETE /identity/v3/users/42ad489d8a654315b8e0baf4b26af26e => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:33.557947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:33.558428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:33.558713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:33.580050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:33.581594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:33.663582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 997/1987] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:33.669463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:33.671541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:33.753022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be7697-f3af-4d5f-a026-5d90b3b34e15 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:33.757843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:33.758881 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:33.760645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:33.760645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:33.777162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:33.779498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:33.792913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 991/1988] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:33.961060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:33.966350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:33.979060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a91016-4259-4aa8-bce5-7a61160bf1ee None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:33.985635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:33.986654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:33.987040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:33.987417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] PATH_INFO: `/v3/projects/896b3030a4a04f74b9d6568bc3abfba0` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:33.989473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] RBAC: Authorizing `identity:delete_project(project_id=896b3030a4a04f74b9d6568bc3abfba0)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:33.992335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:34.030159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:34.121688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '896b3030a4a04f74b9d6568bc3abfba0', 'request_id': 'req-73b0a3cb-acaf-4c38-8685-f6e193784adc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.129118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '896b3030a4a04f74b9d6568bc3abfba0', 'request_id': 'req-73b0a3cb-acaf-4c38-8685-f6e193784adc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.145566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '896b3030a4a04f74b9d6568bc3abfba0', 'request_id': 'req-73b0a3cb-acaf-4c38-8685-f6e193784adc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.269950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] The token cache is being invalidate because project 896b3030a4a04f74b9d6568bc3abfba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:34.270241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.270980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:34.282345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.282871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:34.282951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 998/1989] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:33 2022] DELETE /identity/v3/projects/896b3030a4a04f74b9d6568bc3abfba0 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:34.286407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:34.287760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee0bc14-9510-40e1-842a-aeb4f49f64a4 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:34.287911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:34.349507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:34.350468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:34.350746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:34.351034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] PATH_INFO: `/v3/projects/d6f3fd2a40bd4ffd80662f357c6c3e41` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:34.352257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f3fd2a40bd4ffd80662f357c6c3e41)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:34.354158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:34.356270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b0a3cb-acaf-4c38-8685-f6e193784adc admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:34.361058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:34.361949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:34.362225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:34.362501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] PATH_INFO: `/v3/users/12b5ab491cc44ede929571b6801ccb4a` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:34.364376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12b5ab491cc44ede929571b6801ccb4a)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:34.366320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:34.456170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:34.463699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f3fd2a40bd4ffd80662f357c6c3e41', 'request_id': 'req-c42cb45a-8150-4094-8446-0b2585a77de8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.473288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f3fd2a40bd4ffd80662f357c6c3e41', 'request_id': 'req-c42cb45a-8150-4094-8446-0b2585a77de8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.479282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f3fd2a40bd4ffd80662f357c6c3e41', 'request_id': 'req-c42cb45a-8150-4094-8446-0b2585a77de8'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.532501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] The token cache is being invalidate because project d6f3fd2a40bd4ffd80662f357c6c3e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:34.532916 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.533650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.535962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 992/1990] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:34 2022] DELETE /identity/v3/projects/d6f3fd2a40bd4ffd80662f357c6c3e41 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:34.540927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:34.542649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:34.543730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12b5ab491cc44ede929571b6801ccb4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.557517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12b5ab491cc44ede929571b6801ccb4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.626774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cb45a-8150-4094-8446-0b2585a77de8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:34.632025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:34.633240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:34.633525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:34.633804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] PATH_INFO: `/v3/users/7d663bcc4b8143d49acec5d960dfedae` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:34.633919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12b5ab491cc44ede929571b6801ccb4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.634571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12b5ab491cc44ede929571b6801ccb4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.635059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d663bcc4b8143d49acec5d960dfedae)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:34.635340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12b5ab491cc44ede929571b6801ccb4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.637076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:34.652032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 999/1991] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:34 2022] DELETE /identity/v3/users/12b5ab491cc44ede929571b6801ccb4a => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:34.858612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d663bcc4b8143d49acec5d960dfedae'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.872649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d663bcc4b8143d49acec5d960dfedae'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.954053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d663bcc4b8143d49acec5d960dfedae'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.954716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d663bcc4b8143d49acec5d960dfedae'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.955367 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d663bcc4b8143d49acec5d960dfedae'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:34.969596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 993/1992] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:34 2022] DELETE /identity/v3/users/7d663bcc4b8143d49acec5d960dfedae => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:35.370157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:35.374568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:35.426611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:35.442900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:35.444511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:35.468603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ba3a8-62b6-4d8a-ad44-f7d02c34af97 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:35.537268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:35.555497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:35.556422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:35.556947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:35.557549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] PATH_INFO: `/v3/projects/5f702fca9371466386692e14fb5a77f7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:35.558174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea1dac-7133-4910-bbae-34b45e031f04 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:35.559750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f702fca9371466386692e14fb5a77f7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:35.563451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:35.565744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:35.567799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:35.568354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:35.568843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] PATH_INFO: `/v3/projects/6ba83ed338a6477eb23c978e15731dcb` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:35.570895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ba83ed338a6477eb23c978e15731dcb)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:35.574439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:35.732460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ba83ed338a6477eb23c978e15731dcb', 'request_id': 'req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.737281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f702fca9371466386692e14fb5a77f7', 'request_id': 'req-f7a96a6a-727c-467a-b19e-ab6762bdc93f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.741409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ba83ed338a6477eb23c978e15731dcb', 'request_id': 'req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.746620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f702fca9371466386692e14fb5a77f7', 'request_id': 'req-f7a96a6a-727c-467a-b19e-ab6762bdc93f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.747531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ba83ed338a6477eb23c978e15731dcb', 'request_id': 'req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.751431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f702fca9371466386692e14fb5a77f7', 'request_id': 'req-f7a96a6a-727c-467a-b19e-ab6762bdc93f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.835340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] The token cache is being invalidate because project 6ba83ed338a6477eb23c978e15731dcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:35.835826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.836680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-44ba0e36-649f-4e94-ad72-a10c0bbbdb98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.839155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 994/1993] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:35 2022] DELETE /identity/v3/projects/6ba83ed338a6477eb23c978e15731dcb => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:35.845423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d3f5c-171a-41dc-8125-a8b776393cba None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:35.845685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d3f5c-171a-41dc-8125-a8b776393cba None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:35.845955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d3f5c-171a-41dc-8125-a8b776393cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:35.866879 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7f0d3f5c-171a-41dc-8125-a8b776393cba None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:35.868303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7f0d3f5c-171a-41dc-8125-a8b776393cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:35.890715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] The token cache is being invalidate because project 5f702fca9371466386692e14fb5a77f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:35.891006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.893822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f7a96a6a-727c-467a-b19e-ab6762bdc93f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:35.893822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1000/1994] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:35 2022] DELETE /identity/v3/projects/5f702fca9371466386692e14fb5a77f7 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:35.900025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa270e-5b0b-4c90-bb61-7e16407c4cdd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:35.900629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa270e-5b0b-4c90-bb61-7e16407c4cdd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:35.900928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa270e-5b0b-4c90-bb61-7e16407c4cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:35.927839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d1aa270e-5b0b-4c90-bb61-7e16407c4cdd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:35.929451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d1aa270e-5b0b-4c90-bb61-7e16407c4cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:35.934452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 995/1995] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:35.938178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1001/1996] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:35.946778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941c7d7-9413-4989-853f-55611ead438a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:35.947069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941c7d7-9413-4989-853f-55611ead438a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:35.947348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941c7d7-9413-4989-853f-55611ead438a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:35.950247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b0d2-2402-473f-a09b-00ef9d10d5ad None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:35.950528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b0d2-2402-473f-a09b-00ef9d10d5ad None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:35.950798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b0d2-2402-473f-a09b-00ef9d10d5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:35.966570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b941c7d7-9413-4989-853f-55611ead438a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:35.969896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b941c7d7-9413-4989-853f-55611ead438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:35.971658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3f63b0d2-2402-473f-a09b-00ef9d10d5ad None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:35.972894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3f63b0d2-2402-473f-a09b-00ef9d10d5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:35.976491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 996/1997] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:35.980896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941c7d7-9413-4989-853f-55611ead438a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:35.981430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1002/1998] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:35.985639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63b0d2-2402-473f-a09b-00ef9d10d5ad None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:35.991539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941c7d7-9413-4989-853f-55611ead438a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:35.996033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63b0d2-2402-473f-a09b-00ef9d10d5ad None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:35.996327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:35.997098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:35.997381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:35.997709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:35.998436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:35.998743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.000373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.001355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.002140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.002471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.002821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.003671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:36.004680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.006312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.013583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 997/1999] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:28:36.017340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.018586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1003/2000] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:28:36.027310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a182a4-d757-4767-9363-f837c334e828 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.030835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.032026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.032787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.033066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.033338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.034020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.035952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.040546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81713f59-6ad1-4fcd-bf6f-728861ccfef9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.046581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.047238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.047516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.047816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.048550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.050447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.138732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 998/2001] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:36.142621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.173280 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1004/2002] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:36.179634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.207607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e4c93-15ef-4473-bb5e-f63f26995ce4 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.228652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:36.244421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ad5b-33e0-4548-8200-befcf61bef4d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.249260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.249936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.250223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.250494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.251183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.253193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.264210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.264971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.265322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.265663 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.266435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.268441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.359322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1005/2003] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:36.360189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.361806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:36.366698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:36.375489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 999/2004] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:36.375489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.375720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229710d4-bb4f-43b2-993c-e86725497d54 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.379343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.380243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.380568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.380963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] PATH_INFO: `/v3/users/e82c194591c24998a7812047df6e16aa` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.382176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e82c194591c24998a7812047df6e16aa)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.383536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cced11-97bd-41b0-80b4-5f0d7a1dcc7a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.384164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.388304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.388963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.389243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.389519 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.390274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.391908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.402159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1000/2005] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:36.403334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.413935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fd970-637c-42eb-b94d-92998a8c6107 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.418665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.419312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.419594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.419901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.420616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.422187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.433402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1001/2006] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:36.434204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.443616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eccbee-58f3-48fa-8d65-a97b291ea293 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.448275 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.448912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.449197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.449472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.450207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.451780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.461398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1002/2007] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:36.466070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.471391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d8f3b-ff70-4171-8b6e-0b15b9b23c47 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.476587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.477226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.477509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.477812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.478527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.480118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.493747 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1003/2008] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:36.493747 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.500782 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b410b7-65c1-4dcf-80be-e51f0eaee348 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.505486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.506283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.506565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.506844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] PATH_INFO: `/v3/projects/b5f92ad609ae4d26908bacde07c561d4/users/35f7d76372804881bf091fdfc0ce2e95/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.521732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f92ad609ae4d26908bacde07c561d4, user_id=35f7d76372804881bf091fdfc0ce2e95, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.525178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.626399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e82c194591c24998a7812047df6e16aa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:36.628950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1004/2009] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:36 2022] PUT /identity/v3/projects/b5f92ad609ae4d26908bacde07c561d4/users/35f7d76372804881bf091fdfc0ce2e95/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:36.638817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.649090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e82c194591c24998a7812047df6e16aa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:36.693258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79beb3de-2680-4c66-aa44-67ba7a309ebc admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.698063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfb461-cd06-44bc-a7b0-9309db9ba5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.698960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfb461-cd06-44bc-a7b0-9309db9ba5a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.699242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfb461-cd06-44bc-a7b0-9309db9ba5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.699518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfb461-cd06-44bc-a7b0-9309db9ba5a1 admin admin] PATH_INFO: `/v3/projects/0a97f689104e4884b6e30b0c8606b513/users/6a658b1951a9426d94341940d87b5f95/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.715638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfb461-cd06-44bc-a7b0-9309db9ba5a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a97f689104e4884b6e30b0c8606b513, user_id=6a658b1951a9426d94341940d87b5f95, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.718372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfb461-cd06-44bc-a7b0-9309db9ba5a1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.747672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e82c194591c24998a7812047df6e16aa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:36.748328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e82c194591c24998a7812047df6e16aa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:36.748948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-700384dd-fbd5-4aba-8a95-7b5965d00e55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e82c194591c24998a7812047df6e16aa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:36.766632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1006/2010] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:36 2022] DELETE /identity/v3/users/e82c194591c24998a7812047df6e16aa => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:36.766632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634549-9b07-4744-a78e-82f610a09c0d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.766632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634549-9b07-4744-a78e-82f610a09c0d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.766632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec634549-9b07-4744-a78e-82f610a09c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.793755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ec634549-9b07-4744-a78e-82f610a09c0d None None] MFA Rules not processed for user `35f7d76372804881bf091fdfc0ce2e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:36.795639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ec634549-9b07-4744-a78e-82f610a09c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:36.800270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1005/2011] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:36 2022] PUT /identity/v3/projects/0a97f689104e4884b6e30b0c8606b513/users/6a658b1951a9426d94341940d87b5f95/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:36.805237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.805529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.805813 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.822467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:36.823679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:36.862184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1007/2012] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:36.866952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.867255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.867543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.880830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:36.886520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1006/2013] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:36.887138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.887727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:36.889781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:36.891493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:36.899131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1008/2014] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:36.900245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:36.901604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:36.908302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:36.908535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57934ee3-f350-4937-bf2b-4e9991943d9a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.915114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.915815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.916101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.916488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.917070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.918978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.963371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce89dc90-a252-4a2f-8f2f-cc06a4a7e278 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:36.967933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:36.968593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:36.968913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:36.969158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:36.979209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:36.980766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:36.996868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1009/2015] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:36.997670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.006868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1007/2016] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:36 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:37.007981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.008481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1171da-4b22-4437-a486-0298f3187a2d service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.086262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.086985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.087275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.087732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.098141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84965552-eac4-490b-81d1-4a0331af5f81 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.100788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.102921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.106757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.107690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.108451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.108937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.109887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.112390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.116902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1010/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:37.235144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1008/2018] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:37 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:37.239715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.251078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f869338-acb7-49f7-afeb-6f705537a18f service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.255908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.256574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.256860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.257138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.257860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.259510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.270124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1011/2019] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:37.274703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.284469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04bc01-4f33-4fa0-89ce-bec02320cfca admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.289654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.290857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.290994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.291256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] PATH_INFO: `/v3/projects/887d4c1c59c44575832e98a778e86601/users/6abb8f6ae2f148e299bbca5a6ebad21f/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.308012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887d4c1c59c44575832e98a778e86601, user_id=6abb8f6ae2f148e299bbca5a6ebad21f, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.310651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.388284 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1009/2020] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:37 2022] PUT /identity/v3/projects/887d4c1c59c44575832e98a778e86601/users/6abb8f6ae2f148e299bbca5a6ebad21f/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:37.392597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.403054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed1c35-44c5-4aac-b111-0fcc501f9dc9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.465478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3675ec-9750-45a8-8480-6d7f34baf5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.466145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3675ec-9750-45a8-8480-6d7f34baf5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.466429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3675ec-9750-45a8-8480-6d7f34baf5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.466707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3675ec-9750-45a8-8480-6d7f34baf5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.467436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3675ec-9750-45a8-8480-6d7f34baf5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.469108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3675ec-9750-45a8-8480-6d7f34baf5d2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.480013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1012/2021] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:37.484513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.495326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6e4a46-dc4f-43d2-8e76-2b23af7ea5d9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.500301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.501235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.501631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.501904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] PATH_INFO: `/v3/projects/887d4c1c59c44575832e98a778e86601/users/6abb8f6ae2f148e299bbca5a6ebad21f/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.505051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887d4c1c59c44575832e98a778e86601, user_id=6abb8f6ae2f148e299bbca5a6ebad21f, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.509394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.522592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1010/2022] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:37 2022] PUT /identity/v3/projects/887d4c1c59c44575832e98a778e86601/users/6abb8f6ae2f148e299bbca5a6ebad21f/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:37.532907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.533353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.533634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.563000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] MFA Rules not processed for user `6abb8f6ae2f148e299bbca5a6ebad21f`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:37.564262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:37.588370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:37.613563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.615408 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:37.654097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:37.659377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1013/2023] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:37.671772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.673384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:37.713163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fb4566-3bd2-401c-a7ef-32854986d731 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.718703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.719580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.719908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.720191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] PATH_INFO: `/v3/projects/ba4306524f144654bdfe63b6c1394bac` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.721409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba4306524f144654bdfe63b6c1394bac)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.724138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.739198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e51824-7655-4494-9b0d-7f0ed4373519 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:37.743103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:37.743773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:37.744068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:37.744659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:37.805447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:37.807126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:37.819969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1014/2024] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:37.830211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba4306524f144654bdfe63b6c1394bac', 'request_id': 'req-ffc9025c-202d-46dd-b668-c03da1f13059'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:37.837387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba4306524f144654bdfe63b6c1394bac', 'request_id': 'req-ffc9025c-202d-46dd-b668-c03da1f13059'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:37.849120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba4306524f144654bdfe63b6c1394bac', 'request_id': 'req-ffc9025c-202d-46dd-b668-c03da1f13059'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:37.955168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] The token cache is being invalidate because project ba4306524f144654bdfe63b6c1394bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:37.955832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:37.956181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ffc9025c-202d-46dd-b668-c03da1f13059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:37.957780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1011/2025] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:37 2022] DELETE /identity/v3/projects/ba4306524f144654bdfe63b6c1394bac => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:37.962082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:37.974223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c5a926-5281-4a6e-a39d-a112c118724a service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:38.033960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:38.035633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:38.035965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:38.036237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] PATH_INFO: `/v3/users/813c17bdd3584232b8d08645cc501fd7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:38.037527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813c17bdd3584232b8d08645cc501fd7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:38.039460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:38.062662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:38.276945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813c17bdd3584232b8d08645cc501fd7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:38.308611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813c17bdd3584232b8d08645cc501fd7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:38.399438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813c17bdd3584232b8d08645cc501fd7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:38.400104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813c17bdd3584232b8d08645cc501fd7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:38.400711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813c17bdd3584232b8d08645cc501fd7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:38.414423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1015/2026] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:37 2022] DELETE /identity/v3/users/813c17bdd3584232b8d08645cc501fd7 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:38.610590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:38.611866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:38.612404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:38.659144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] MFA Rules not processed for user `6a658b1951a9426d94341940d87b5f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:38.661206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:38.734712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1012/2027] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:38.749099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:38.750666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:38.817618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a395e-4998-4a26-afa0-d2562fd62524 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:38.821832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0f064-7576-439a-a462-d4c186378c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:38.822500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0f064-7576-439a-a462-d4c186378c42 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:38.822782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0f064-7576-439a-a462-d4c186378c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:38.823058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0f064-7576-439a-a462-d4c186378c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:38.833319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0f064-7576-439a-a462-d4c186378c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:38.834877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0f064-7576-439a-a462-d4c186378c42 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:38.845572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1016/2028] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:38.988894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:38.990391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:39.033755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:39.057299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc73c79-bb52-40bc-bc09-c27478209988 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:39.062000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:39.062874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:39.063157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:39.063438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] PATH_INFO: `/v3/projects/fef7fb75ca6744369ecf3883626f6ace` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:39.064699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef7fb75ca6744369ecf3883626f6ace)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:39.066690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:39.199605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef7fb75ca6744369ecf3883626f6ace', 'request_id': 'req-8fbbf2a9-1526-47bb-966b-393642e8e0e1'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:39.205850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef7fb75ca6744369ecf3883626f6ace', 'request_id': 'req-8fbbf2a9-1526-47bb-966b-393642e8e0e1'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:39.211845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef7fb75ca6744369ecf3883626f6ace', 'request_id': 'req-8fbbf2a9-1526-47bb-966b-393642e8e0e1'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:39.320212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] The token cache is being invalidate because project fef7fb75ca6744369ecf3883626f6ace was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:39.320513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:39.321303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8fbbf2a9-1526-47bb-966b-393642e8e0e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:39.323727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1013/2029] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:38 2022] DELETE /identity/v3/projects/fef7fb75ca6744369ecf3883626f6ace => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:39.330195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07133ab-ab6a-4513-93b7-2a127b335b8a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:39.330516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07133ab-ab6a-4513-93b7-2a127b335b8a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:39.330901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07133ab-ab6a-4513-93b7-2a127b335b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:39.349775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f07133ab-ab6a-4513-93b7-2a127b335b8a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:39.351022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f07133ab-ab6a-4513-93b7-2a127b335b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:39.414342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1017/2030] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:39.422359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82ea9f-09eb-41d7-b89d-f409d478edd1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:39.422800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82ea9f-09eb-41d7-b89d-f409d478edd1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:39.423079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82ea9f-09eb-41d7-b89d-f409d478edd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:39.443684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5e82ea9f-09eb-41d7-b89d-f409d478edd1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:39.451824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5e82ea9f-09eb-41d7-b89d-f409d478edd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:39.453099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1014/2031] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:39.458138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07133ab-ab6a-4513-93b7-2a127b335b8a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:39.467975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07133ab-ab6a-4513-93b7-2a127b335b8a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:39.473495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:39.474171 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:39.474461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:39.474739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:39.475474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:39.475808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:39.477450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:39.491379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1018/2032] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:28:39.496164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82ea9f-09eb-41d7-b89d-f409d478edd1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:39.506586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82ea9f-09eb-41d7-b89d-f409d478edd1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:39.512089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:39.512751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:39.513662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:39.513662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:39.514260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:39.516232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:39.710825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1015/2033] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:39 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:39.716207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:39.726903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a721-7726-4078-bdc3-ab99ab73e53c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:39.747993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:39.787818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:39.788510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:39.788795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:39.789092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:39.789794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:39.791890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:39.932484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1019/2034] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:39 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:39.937362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:39.951033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9859c7d9-e57f-4cc3-b79f-3907684fa3cb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:39.956427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:39.957651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:39.957967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:39.958236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:39.962474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:39.962474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:39.980490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1016/2035] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:39.984425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:39.995099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8a148e-122b-40aa-a128-1ebce97ca375 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:40.000013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a16e27-5512-49ba-9c50-a2dbaed83f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:40.000674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a16e27-5512-49ba-9c50-a2dbaed83f47 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.000954 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a16e27-5512-49ba-9c50-a2dbaed83f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.001228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a16e27-5512-49ba-9c50-a2dbaed83f47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.001948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a16e27-5512-49ba-9c50-a2dbaed83f47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:40.003552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a16e27-5512-49ba-9c50-a2dbaed83f47 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:40.014952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1020/2036] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:40.023030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:40.033015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ff71c6-2658-44b9-9cb1-6bde36fc7679 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:40.039051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:40.039979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.042883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.042883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] PATH_INFO: `/v3/projects/462a8ac9f5d84f91b6173daf0acab971/users/af1171c8d2514721ad9a2815a9dbb5fb/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.057293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=462a8ac9f5d84f91b6173daf0acab971, user_id=af1171c8d2514721ad9a2815a9dbb5fb, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:40.059957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:40.164312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1017/2037] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:40 2022] PUT /identity/v3/projects/462a8ac9f5d84f91b6173daf0acab971/users/af1171c8d2514721ad9a2815a9dbb5fb/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:40.175472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c74be-dcb7-4bd9-893b-08331cf9049a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.175819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c74be-dcb7-4bd9-893b-08331cf9049a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.176106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c74be-dcb7-4bd9-893b-08331cf9049a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.204179 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-af2c74be-dcb7-4bd9-893b-08331cf9049a None None] MFA Rules not processed for user `af1171c8d2514721ad9a2815a9dbb5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:40.205385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-af2c74be-dcb7-4bd9-893b-08331cf9049a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:40.267452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1021/2038] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:40.272495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:40.333475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62ff6c-eeca-4aaf-bba0-976da985e1ad admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:40.338303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:40.338964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.339243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.339518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.340244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:40.342130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:40.350058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:40.504606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1018/2039] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:40 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:40.508744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c74be-dcb7-4bd9-893b-08331cf9049a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:40.519800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c74be-dcb7-4bd9-893b-08331cf9049a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:40.583211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:40.584181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.584470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.584863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.586783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:40.592039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:40.760304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1022/2040] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:40 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:40.765574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:40.775871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6cc805-822e-43a6-a2d2-eadc975c60cd admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:40.780289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:40.781002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.781285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.781567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.782308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:40.783938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:40.794356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1019/2041] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:40.798260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:40.809092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79bd730-9d26-4005-935e-a3a5e64eef17 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:40.813695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432103d-6091-47a6-a052-4b8a8664d898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:40.814350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432103d-6091-47a6-a052-4b8a8664d898 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.814634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432103d-6091-47a6-a052-4b8a8664d898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.814906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432103d-6091-47a6-a052-4b8a8664d898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.815656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432103d-6091-47a6-a052-4b8a8664d898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:40.817919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432103d-6091-47a6-a052-4b8a8664d898 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:40.828003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1023/2042] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:40.832936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:40.843066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d77e18-930d-4216-8528-b0391d3eb12a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:40.848097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:40.848972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.849254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.849529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] PATH_INFO: `/v3/projects/13f39cea2ede4da997ad0cb4392700a4/users/610db21d414141569801d203a62054a4/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.864857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=13f39cea2ede4da997ad0cb4392700a4, user_id=610db21d414141569801d203a62054a4, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:40.867480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:40.946776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1020/2043] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:40 2022] PUT /identity/v3/projects/13f39cea2ede4da997ad0cb4392700a4/users/610db21d414141569801d203a62054a4/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:40.956862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fb23-017d-419c-bc93-6ef14a994b98 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:40.957145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fb23-017d-419c-bc93-6ef14a994b98 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:40.957419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8fb23-017d-419c-bc93-6ef14a994b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:40.976466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:40.983724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-72f8fb23-017d-419c-bc93-6ef14a994b98 None None] MFA Rules not processed for user `610db21d414141569801d203a62054a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:40.984950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-72f8fb23-017d-419c-bc93-6ef14a994b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:41.059380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1024/2044] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:41.063986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:41.126258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193acb9a-d7a7-43b7-9dba-6c5ad5f608ae admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:41.131649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:41.132336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.132618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.132893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.133591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:41.135473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:41.223250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1021/2045] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:41 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:41.227380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8fb23-017d-419c-bc93-6ef14a994b98 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:41.238683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8fb23-017d-419c-bc93-6ef14a994b98 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:41.294858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:41.295441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.295725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.296026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.296720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:41.298625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:41.416888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1025/2046] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:41 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:41.420963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:41.432000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9dfcf-479c-4070-819f-5420efa91ed3 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:41.436757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:41.437422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.437702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.442925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.442925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:41.442925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:41.451649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1022/2047] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:41.455875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:41.466111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3316c82-8780-41f0-be97-c0dfba8d3155 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:41.470744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:41.471670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.471970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.472249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] PATH_INFO: `/v3/projects/a89ca7e9f61b4e239bf8d3633a2441be/users/8d9550e11b404588a007e133db1bd83b/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.487355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89ca7e9f61b4e239bf8d3633a2441be, user_id=8d9550e11b404588a007e133db1bd83b, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:41.490739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:41.595855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1026/2048] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:41 2022] PUT /identity/v3/projects/a89ca7e9f61b4e239bf8d3633a2441be/users/8d9550e11b404588a007e133db1bd83b/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:41.600662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:41.611138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08759a-f416-4738-af1f-acefaa41dc1c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:41.676452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8be16-ce47-4172-8a3f-73fa40152775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:41.676784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8be16-ce47-4172-8a3f-73fa40152775 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.677064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8be16-ce47-4172-8a3f-73fa40152775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.677336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8be16-ce47-4172-8a3f-73fa40152775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.678336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8be16-ce47-4172-8a3f-73fa40152775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:41.680020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8be16-ce47-4172-8a3f-73fa40152775 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:41.691223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1023/2049] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:41.695668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:41.718085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8b5bd-81f6-4ab5-adca-0578a5764b41 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:41.724175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:41.725409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.725827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.726109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] PATH_INFO: `/v3/projects/a89ca7e9f61b4e239bf8d3633a2441be/users/8d9550e11b404588a007e133db1bd83b/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.730011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a89ca7e9f61b4e239bf8d3633a2441be, user_id=8d9550e11b404588a007e133db1bd83b, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:41.732863 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:41.745615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1027/2050] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:41 2022] PUT /identity/v3/projects/a89ca7e9f61b4e239bf8d3633a2441be/users/8d9550e11b404588a007e133db1bd83b/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:41.758645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d600a25-276d-4220-9941-948efacfa78d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.758935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d600a25-276d-4220-9941-948efacfa78d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.759511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d600a25-276d-4220-9941-948efacfa78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.790568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-0d600a25-276d-4220-9941-948efacfa78d None None] MFA Rules not processed for user `8d9550e11b404588a007e133db1bd83b`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:41.791983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0d600a25-276d-4220-9941-948efacfa78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:41.806049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:41.866959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1024/2051] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:41.877869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:41.879662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:41.942822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6523319-1b96-41c7-8702-052cceb985f8 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:41.946941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:41.947674 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:41.948067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:41.948421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:41.955821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:41.958892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:41.960371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:41.970716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1028/2052] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:42.044564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d600a25-276d-4220-9941-948efacfa78d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:42.045958 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0d600a25-276d-4220-9941-948efacfa78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:42.058302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d600a25-276d-4220-9941-948efacfa78d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:42.061996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adbac73-8186-4ea9-9609-4fcf43dc55c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:42.062643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbac73-8186-4ea9-9609-4fcf43dc55c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:42.062926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbac73-8186-4ea9-9609-4fcf43dc55c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:42.063201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adbac73-8186-4ea9-9609-4fcf43dc55c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:42.122097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbac73-8186-4ea9-9609-4fcf43dc55c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:42.123646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adbac73-8186-4ea9-9609-4fcf43dc55c1 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:42.132938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1025/2053] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:47.024208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:47.025791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.042204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:47.042680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:47.042926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:47.062939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] MFA Rules not processed for user `6a658b1951a9426d94341940d87b5f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:47.064266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.101249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbff925-8251-4f23-bfa3-5b7d23b7c6b4 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:47.107113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:47.107113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:47.107676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:47.107964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] PATH_INFO: `/v3/users/71302686cd59424ba952b49c0fad3577` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:47.109414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71302686cd59424ba952b49c0fad3577)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:47.111134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:47.128697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1026/2054] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:47.299478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:47.301029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.329183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d808564-2d71-4525-aa86-8c9923b86a51 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:47.335618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:47.336853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:47.337162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:47.337611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] PATH_INFO: `/v3/users/35f7d76372804881bf091fdfc0ce2e95` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:47.338945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=35f7d76372804881bf091fdfc0ce2e95)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:47.341035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:47.346336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71302686cd59424ba952b49c0fad3577'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.364138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71302686cd59424ba952b49c0fad3577'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.439254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71302686cd59424ba952b49c0fad3577'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.440053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71302686cd59424ba952b49c0fad3577'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.440986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c4c764d8-2cfc-4fb7-99d2-532d15c62e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71302686cd59424ba952b49c0fad3577'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.461980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1029/2055] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:47 2022] DELETE /identity/v3/users/71302686cd59424ba952b49c0fad3577 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:47.596047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35f7d76372804881bf091fdfc0ce2e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.597640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:47.616338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35f7d76372804881bf091fdfc0ce2e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.716820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35f7d76372804881bf091fdfc0ce2e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.716820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35f7d76372804881bf091fdfc0ce2e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.716820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35f7d76372804881bf091fdfc0ce2e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:47.741669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1027/2056] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:47 2022] DELETE /identity/v3/users/35f7d76372804881bf091fdfc0ce2e95 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:47.745105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:47.745399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:47.745681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:47.770572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:47.771872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.817480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:47.818891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.854109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1030/2057] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:47.863737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:47.865239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:47.893968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:47.906522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48425dcb-a121-4578-ab37-00a442ee6a7f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:47.912368 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:47.913456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:47.913744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:47.914024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] PATH_INFO: `/v3/projects/199159bff7e5433ca3da168465c2e22c` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:47.915282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=199159bff7e5433ca3da168465c2e22c)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:47.917384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:47.942206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da77c19-5a99-4fbf-aeff-03eec525526c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:47.946112 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46e240-1b07-4150-810d-0a73c80cd4f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:47.946920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46e240-1b07-4150-810d-0a73c80cd4f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:47.947259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46e240-1b07-4150-810d-0a73c80cd4f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:47.951951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46e240-1b07-4150-810d-0a73c80cd4f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:47.958688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46e240-1b07-4150-810d-0a73c80cd4f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:47.960381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46e240-1b07-4150-810d-0a73c80cd4f9 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:47.972882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1031/2058] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:28:48.078685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199159bff7e5433ca3da168465c2e22c', 'request_id': 'req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.086481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199159bff7e5433ca3da168465c2e22c', 'request_id': 'req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.094175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199159bff7e5433ca3da168465c2e22c', 'request_id': 'req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.206532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] The token cache is being invalidate because project 199159bff7e5433ca3da168465c2e22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:48.206532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.206532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-de6ee4be-d0ed-48af-b3bb-7b6439a5f225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.206532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1028/2059] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:47 2022] DELETE /identity/v3/projects/199159bff7e5433ca3da168465c2e22c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:48.208700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122332ec-1670-4895-aad1-f55951a68532 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.209197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122332ec-1670-4895-aad1-f55951a68532 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.209417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122332ec-1670-4895-aad1-f55951a68532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.248471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-122332ec-1670-4895-aad1-f55951a68532 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:48.250292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-122332ec-1670-4895-aad1-f55951a68532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:48.321449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1032/2060] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:48.329262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94a3cc-817b-435e-885d-34bb336a217f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.329556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94a3cc-817b-435e-885d-34bb336a217f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.329836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94a3cc-817b-435e-885d-34bb336a217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.348891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1f94a3cc-817b-435e-885d-34bb336a217f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:48.350099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1f94a3cc-817b-435e-885d-34bb336a217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:48.358370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1029/2061] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:48.362412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122332ec-1670-4895-aad1-f55951a68532 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.373323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122332ec-1670-4895-aad1-f55951a68532 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.378438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.379096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.379375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.379757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.380672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:48.380959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.382891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:48.395512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1033/2062] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:28:48.400157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94a3cc-817b-435e-885d-34bb336a217f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.409324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94a3cc-817b-435e-885d-34bb336a217f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.414581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.415232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.415517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.415829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.416518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.418403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:48.457899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.460130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:48.475281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c6dbb-3446-4332-8ff0-1773e9b4181f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.481814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.483511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.483850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.484097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] PATH_INFO: `/v3/projects/b5f92ad609ae4d26908bacde07c561d4` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.496504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f92ad609ae4d26908bacde07c561d4)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.496504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:48.546758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1030/2063] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:48 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:48.552461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.585256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:48.625083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb16fad-8f65-4ba6-9d7e-a0578647ea5e admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.633105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f92ad609ae4d26908bacde07c561d4', 'request_id': 'req-fc10d5a4-e96e-413d-928e-50c733381670'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.640886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f92ad609ae4d26908bacde07c561d4', 'request_id': 'req-fc10d5a4-e96e-413d-928e-50c733381670'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.647168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f92ad609ae4d26908bacde07c561d4', 'request_id': 'req-fc10d5a4-e96e-413d-928e-50c733381670'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.685030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.685714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.686049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.686356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.687056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.689397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:48.715068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] The token cache is being invalidate because project b5f92ad609ae4d26908bacde07c561d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:48.715068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.715068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:48.715068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1034/2064] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:48 2022] DELETE /identity/v3/projects/b5f92ad609ae4d26908bacde07c561d4 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:48.716644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.718304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:48.731658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc10d5a4-e96e-413d-928e-50c733381670 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.736700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.737672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.737986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.738269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] PATH_INFO: `/v3/users/6abb8f6ae2f148e299bbca5a6ebad21f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.741396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abb8f6ae2f148e299bbca5a6ebad21f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.741684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:48.757212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:48.809629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1031/2065] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:48 2022] POST /identity/v3/users => generated 291 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:48.816174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.819355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:48.836170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce0b8be-309b-43ca-84ee-4d3607ccf788 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.841107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.841983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.842191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.842473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.843255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.845054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:48.859997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1032/2066] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:48.861805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.873423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39b89b-fb91-47ca-9d16-17dfaf179b65 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.878713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.879393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.879679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.879997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.880737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.882536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:48.901192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1033/2067] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:48.913066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:48.924141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbccb44-c296-4d91-9074-a5a0dfc1445f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:48.929893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b72fad-b234-4519-8d90-7f7a826adbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:48.930797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b72fad-b234-4519-8d90-7f7a826adbd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:48.931090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b72fad-b234-4519-8d90-7f7a826adbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:48.931344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b72fad-b234-4519-8d90-7f7a826adbd7 admin admin] PATH_INFO: `/v3/projects/523b7629c65848ffbc71ef954b7f9ddc/users/9858a1dcd4754d8c99b86ccfd2c8a359/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:48.949745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b72fad-b234-4519-8d90-7f7a826adbd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523b7629c65848ffbc71ef954b7f9ddc, user_id=9858a1dcd4754d8c99b86ccfd2c8a359, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:48.952587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b72fad-b234-4519-8d90-7f7a826adbd7 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.012028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abb8f6ae2f148e299bbca5a6ebad21f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.025420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abb8f6ae2f148e299bbca5a6ebad21f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.033912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1034/2068] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:48 2022] PUT /identity/v3/projects/523b7629c65848ffbc71ef954b7f9ddc/users/9858a1dcd4754d8c99b86ccfd2c8a359/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:49.050814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e680c-a8a7-416c-b11c-e864f7bd890f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.051102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e680c-a8a7-416c-b11c-e864f7bd890f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.051437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e680c-a8a7-416c-b11c-e864f7bd890f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.079357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ca1e680c-a8a7-416c-b11c-e864f7bd890f None None] MFA Rules not processed for user `9858a1dcd4754d8c99b86ccfd2c8a359`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:49.080637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ca1e680c-a8a7-416c-b11c-e864f7bd890f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:49.146757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1035/2069] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:49.148618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abb8f6ae2f148e299bbca5a6ebad21f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.149316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abb8f6ae2f148e299bbca5a6ebad21f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.150368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abb8f6ae2f148e299bbca5a6ebad21f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.151179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e680c-a8a7-416c-b11c-e864f7bd890f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:49.164180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1035/2070] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:48 2022] DELETE /identity/v3/users/6abb8f6ae2f148e299bbca5a6ebad21f => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:28:49.217239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e680c-a8a7-416c-b11c-e864f7bd890f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:49.278322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:49.278987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.279265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.279558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.280289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:49.282166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.403534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1036/2071] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:49 2022] POST /identity/v3/projects => generated 381 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:49.407425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:49.409302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:49.422545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be04489-d2ad-47d2-94a7-5120f4912a46 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:49.428291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:49.484192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:49.485245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.485750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.487025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.487802 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:49.490345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.583780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1036/2072] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:49 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:49.587707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:49.599376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a81f1e7-7578-40df-a7c1-61be2c2e4c99 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:49.604182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:49.604950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.605105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.605382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.606037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:49.607817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.614485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:49.616616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:49.625229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1037/2073] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:49.625830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:49.637065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e54df0-5c27-4b9c-b7c9-281960c58aa0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:49.643075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:49.643545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.644119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.644467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] PATH_INFO: `/v3/projects/c0486ad03b12460b9a278a8b1e4087e4/users/1859545f62684fa29a103fa0d0ca69d9/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.648179 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeeee7-e443-4000-8770-34b8e9ff16db admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:49.654268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:49.655175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.655424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.655705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] PATH_INFO: `/v3/projects/887d4c1c59c44575832e98a778e86601` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.657527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=887d4c1c59c44575832e98a778e86601)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:49.660085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.666731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0486ad03b12460b9a278a8b1e4087e4, user_id=1859545f62684fa29a103fa0d0ca69d9, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:49.670904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.801492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1038/2074] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:49 2022] PUT /identity/v3/projects/c0486ad03b12460b9a278a8b1e4087e4/users/1859545f62684fa29a103fa0d0ca69d9/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:49.802451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:49.804598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:49.823136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887d4c1c59c44575832e98a778e86601', 'request_id': 'req-f3a27c08-0e94-4cd3-a24c-7047f86563a7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.831961 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887d4c1c59c44575832e98a778e86601', 'request_id': 'req-f3a27c08-0e94-4cd3-a24c-7047f86563a7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.838811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887d4c1c59c44575832e98a778e86601', 'request_id': 'req-f3a27c08-0e94-4cd3-a24c-7047f86563a7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.875487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544fd355-5348-44d1-81e3-660a9b61f8f8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:49.884737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:49.935241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:49.936179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.936314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] The token cache is being invalidate because project 887d4c1c59c44575832e98a778e86601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:49.936484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.936583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.936738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] PATH_INFO: `/v3/users/af1171c8d2514721ad9a2815a9dbb5fb` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.937315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:49.937919 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1171c8d2514721ad9a2815a9dbb5fb)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:49.938898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1037/2075] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:49 2022] DELETE /identity/v3/projects/887d4c1c59c44575832e98a778e86601 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:49.939664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:49.939884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.941061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:49.950524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=79,keystone:DELETE=16,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:49.954888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27c08-0e94-4cd3-a24c-7047f86563a7 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:49.959795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd43b03a-3c35-4263-86dd-a92d4418454d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:49.960470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43b03a-3c35-4263-86dd-a92d4418454d admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.960775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43b03a-3c35-4263-86dd-a92d4418454d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.961049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd43b03a-3c35-4263-86dd-a92d4418454d admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.961781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43b03a-3c35-4263-86dd-a92d4418454d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:49.963474 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd43b03a-3c35-4263-86dd-a92d4418454d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:49.975707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1038/2076] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:49.977303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1444fe-4dc3-432b-bae5-2adcc69eb06e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:49.977594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1444fe-4dc3-432b-bae5-2adcc69eb06e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:49.977869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1444fe-4dc3-432b-bae5-2adcc69eb06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:49.996629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1c1444fe-4dc3-432b-bae5-2adcc69eb06e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:49.997843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1c1444fe-4dc3-432b-bae5-2adcc69eb06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.005854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1039/2077] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:50.006932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1444fe-4dc3-432b-bae5-2adcc69eb06e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.017434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1444fe-4dc3-432b-bae5-2adcc69eb06e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.022834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff09dc-7cee-4b67-a3f8-d43eeadff79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.023775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff09dc-7cee-4b67-a3f8-d43eeadff79d admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.024076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff09dc-7cee-4b67-a3f8-d43eeadff79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.024370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff09dc-7cee-4b67-a3f8-d43eeadff79d admin admin] PATH_INFO: `/v3/projects/c0486ad03b12460b9a278a8b1e4087e4/users/1859545f62684fa29a103fa0d0ca69d9/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.026435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff09dc-7cee-4b67-a3f8-d43eeadff79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0486ad03b12460b9a278a8b1e4087e4, user_id=1859545f62684fa29a103fa0d0ca69d9, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.029133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff09dc-7cee-4b67-a3f8-d43eeadff79d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.052813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1040/2078] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:50 2022] PUT /identity/v3/projects/c0486ad03b12460b9a278a8b1e4087e4/users/1859545f62684fa29a103fa0d0ca69d9/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:50.054202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20b1d-8f29-4e0c-a579-ec73f8f9bccf None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.054497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20b1d-8f29-4e0c-a579-ec73f8f9bccf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.054785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe20b1d-8f29-4e0c-a579-ec73f8f9bccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.081778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ffe20b1d-8f29-4e0c-a579-ec73f8f9bccf None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:50.083102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ffe20b1d-8f29-4e0c-a579-ec73f8f9bccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.165889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1041/2079] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:50.167208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8083e0-b5ec-4370-83f3-30e72f46af64 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.167566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8083e0-b5ec-4370-83f3-30e72f46af64 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.167939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8083e0-b5ec-4370-83f3-30e72f46af64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.203571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1171c8d2514721ad9a2815a9dbb5fb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:50.229988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1171c8d2514721ad9a2815a9dbb5fb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:50.236680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ef8083e0-b5ec-4370-83f3-30e72f46af64 None None] MFA Rules not processed for user `1859545f62684fa29a103fa0d0ca69d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:50.239384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ef8083e0-b5ec-4370-83f3-30e72f46af64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.307846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1171c8d2514721ad9a2815a9dbb5fb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:50.308528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1171c8d2514721ad9a2815a9dbb5fb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:50.312214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1171c8d2514721ad9a2815a9dbb5fb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:50.321522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1042/2080] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:50.322536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8083e0-b5ec-4370-83f3-30e72f46af64 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.335621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8083e0-b5ec-4370-83f3-30e72f46af64 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.341083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b0237-a114-41fa-9f09-8c012643673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.341757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0237-a114-41fa-9f09-8c012643673f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.342038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0237-a114-41fa-9f09-8c012643673f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.342316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b0237-a114-41fa-9f09-8c012643673f admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.346136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b0237-a114-41fa-9f09-8c012643673f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:50.346298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1039/2081] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:49 2022] DELETE /identity/v3/users/af1171c8d2514721ad9a2815a9dbb5fb => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:50.346402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b0237-a114-41fa-9f09-8c012643673f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.346478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.346576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163b0237-a114-41fa-9f09-8c012643673f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.348188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.361071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1043/2082] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:28:50.366144 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.366515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.366727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.387552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:50.388086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.415691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81e82-408d-4476-b656-de0651d78ea6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.419438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.420130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.420414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.420687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.422541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.457402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:50.467432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1044/2083] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:50.468723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.470277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.479477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:50.495212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c34c9-0a1e-4fdc-8378-47c098938540 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.500289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.500979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.501412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.507226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.507226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.507226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.507479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.507479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.520833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1040/2084] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:50.521949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.523423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.586439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1045/2085] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:50 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:50.587679 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.595561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fce9e-255e-496d-8030-55d696d1947d service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.656202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.656961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.657312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.657648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.668615 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.671106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.678509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ad49-c48c-4142-a214-2756ed1f880a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.685831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1041/2086] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:50.686757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.687503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.687831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.688097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.689032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.693190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.696406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.767190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1046/2087] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:50 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:50.772856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.781034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6fd842-bb13-468d-87ab-be41244eca15 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.784600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dabb9d-9118-452c-9d82-1f460d996989 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.785969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d5456-f3c5-47bb-a335-805f1c4ef89c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.786626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d5456-f3c5-47bb-a335-805f1c4ef89c service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.786910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d5456-f3c5-47bb-a335-805f1c4ef89c service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.787199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d5456-f3c5-47bb-a335-805f1c4ef89c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.789609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-db7d5456-f3c5-47bb-a335-805f1c4ef89c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:50.791374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.792259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.792988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.793444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.794459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.796734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.816894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1047/2088] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:50.818067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.833457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52dc04-3d79-496c-9ff2-9b5ac2277df9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.847785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.847785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.847785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.847785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.848348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.848348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.859646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1048/2089] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:50.860745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:50.863943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d5456-f3c5-47bb-a335-805f1c4ef89c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.865847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d5456-f3c5-47bb-a335-805f1c4ef89c service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:50.871914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afd69c-2f39-45d9-974c-28e2cf6507f3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:50.877890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:50.879335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:50.879685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:50.880088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] PATH_INFO: `/v3/projects/1ec4faf08b8a4cd6826a94616e61437a/users/e8df94da49dd4619a4420905934af4c7/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:50.884160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1042/2090] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:50.898123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ec4faf08b8a4cd6826a94616e61437a, user_id=e8df94da49dd4619a4420905934af4c7, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:50.900993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:51.002164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1049/2091] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:50 2022] PUT /identity/v3/projects/1ec4faf08b8a4cd6826a94616e61437a/users/e8df94da49dd4619a4420905934af4c7/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:51.006398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:51.006690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:51.006972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:51.043313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] MFA Rules not processed for user `e8df94da49dd4619a4420905934af4c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:51.044789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:51.122035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1043/2092] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:51.133437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:51.135331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:51.198864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:51.201441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:51.220371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f27be-85bf-427e-855b-49ca68efb787 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:51.221444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:51.222109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:51.222609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:51.222886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:51.246605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:51.248743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:51.262359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1050/2093] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:51.316019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c475c-4ede-4fde-a36d-dfdbd5deac71 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:51.323865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:51.324911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:51.325286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:51.325629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] PATH_INFO: `/v3/projects/462a8ac9f5d84f91b6173daf0acab971` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:51.327311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=462a8ac9f5d84f91b6173daf0acab971)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:51.329569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:51.524668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '462a8ac9f5d84f91b6173daf0acab971', 'request_id': 'req-41cba096-307d-44f2-9ab4-cd894d80d396'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:51.538256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:51.543653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '462a8ac9f5d84f91b6173daf0acab971', 'request_id': 'req-41cba096-307d-44f2-9ab4-cd894d80d396'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:51.548010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '462a8ac9f5d84f91b6173daf0acab971', 'request_id': 'req-41cba096-307d-44f2-9ab4-cd894d80d396'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:51.686909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] The token cache is being invalidate because project 462a8ac9f5d84f91b6173daf0acab971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:51.686909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:51.686909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:51.686909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1044/2094] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:51 2022] DELETE /identity/v3/projects/462a8ac9f5d84f91b6173daf0acab971 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:51.689100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:51.700758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e3f8d5-3427-46f8-9b22-ee3871ae9a83 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:51.710700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:51.759449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:51.760412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:51.760694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:51.760962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] PATH_INFO: `/v3/users/610db21d414141569801d203a62054a4` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:51.762192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=610db21d414141569801d203a62054a4)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:51.764288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:51.948412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '610db21d414141569801d203a62054a4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:51.960606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '610db21d414141569801d203a62054a4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.067206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '610db21d414141569801d203a62054a4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.068240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '610db21d414141569801d203a62054a4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.069031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '610db21d414141569801d203a62054a4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.085354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1051/2095] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:51 2022] DELETE /identity/v3/users/610db21d414141569801d203a62054a4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:52.544499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:52.546174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:52.571215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cba096-307d-44f2-9ab4-cd894d80d396 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:52.600097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:52.675832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:52.677074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:52.677508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:52.678224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] PATH_INFO: `/v3/projects/13f39cea2ede4da997ad0cb4392700a4` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:52.680605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13f39cea2ede4da997ad0cb4392700a4)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:52.681857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:52.822141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f39cea2ede4da997ad0cb4392700a4', 'request_id': 'req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.829815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f39cea2ede4da997ad0cb4392700a4', 'request_id': 'req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.835836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f39cea2ede4da997ad0cb4392700a4', 'request_id': 'req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.932360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] The token cache is being invalidate because project 13f39cea2ede4da997ad0cb4392700a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:52.932902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.933708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:52.941135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1045/2096] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:52 2022] DELETE /identity/v3/projects/13f39cea2ede4da997ad0cb4392700a4 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:52.945025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:52.947578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:52.961739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715ba76-d230-4400-96ce-ac68115a7e68 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:53.020916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:53.021731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:53.022012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:53.022292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] PATH_INFO: `/v3/users/8d9550e11b404588a007e133db1bd83b` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:53.024412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9550e11b404588a007e133db1bd83b)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:53.025576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:53.229032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:53.242116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9550e11b404588a007e133db1bd83b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:53.260705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9550e11b404588a007e133db1bd83b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:53.359049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9550e11b404588a007e133db1bd83b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:53.359936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9550e11b404588a007e133db1bd83b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:53.367849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8910ecbd-503a-4058-8a9e-f610bcb8dffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9550e11b404588a007e133db1bd83b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:53.373707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1052/2097] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:52 2022] DELETE /identity/v3/users/8d9550e11b404588a007e133db1bd83b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:53.875014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:53.877242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:53.892106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d71cd5-b99a-4738-8b00-558b7c5fc17a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:54.001426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:54.002320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.002605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.002882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] PATH_INFO: `/v3/projects/a89ca7e9f61b4e239bf8d3633a2441be` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.008769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a89ca7e9f61b4e239bf8d3633a2441be)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:54.010714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:54.145198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:54.150165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a89ca7e9f61b4e239bf8d3633a2441be', 'request_id': 'req-f09cd11a-7d52-41fe-892e-ca9773253ed3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:54.162214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a89ca7e9f61b4e239bf8d3633a2441be', 'request_id': 'req-f09cd11a-7d52-41fe-892e-ca9773253ed3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:54.171560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a89ca7e9f61b4e239bf8d3633a2441be', 'request_id': 'req-f09cd11a-7d52-41fe-892e-ca9773253ed3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:54.261920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] The token cache is being invalidate because project a89ca7e9f61b4e239bf8d3633a2441be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:54.262339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:54.269974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f09cd11a-7d52-41fe-892e-ca9773253ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:54.269974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1046/2098] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:53 2022] DELETE /identity/v3/projects/a89ca7e9f61b4e239bf8d3633a2441be => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:54.272316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11899adf-6104-4757-a350-ad0a9ffe8711 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.272793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11899adf-6104-4757-a350-ad0a9ffe8711 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.272867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11899adf-6104-4757-a350-ad0a9ffe8711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.292506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-11899adf-6104-4757-a350-ad0a9ffe8711 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:54.293773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-11899adf-6104-4757-a350-ad0a9ffe8711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:54.364703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1053/2099] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:54.373347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fd6b-8bc2-4d1d-a6d0-f71a72e74f3a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.373785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fd6b-8bc2-4d1d-a6d0-f71a72e74f3a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.374065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac8fd6b-8bc2-4d1d-a6d0-f71a72e74f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.393262 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1ac8fd6b-8bc2-4d1d-a6d0-f71a72e74f3a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:54.395033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1ac8fd6b-8bc2-4d1d-a6d0-f71a72e74f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:54.403637 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1047/2100] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:54.407912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11899adf-6104-4757-a350-ad0a9ffe8711 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:54.418146 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11899adf-6104-4757-a350-ad0a9ffe8711 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:54.424728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:54.425439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.425720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.426045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.426849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:54.427227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:54.429172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:54.446760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1054/2101] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:28:54.448024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8fd6b-8bc2-4d1d-a6d0-f71a72e74f3a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:54.459166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac8fd6b-8bc2-4d1d-a6d0-f71a72e74f3a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:54.464581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:54.465488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.465849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.466134 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.466833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:54.468816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:54.609406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1048/2102] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:54 2022] POST /identity/v3/projects => generated 371 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:54.610557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:54.620757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd66098-9fae-47f2-8a10-332decc93990 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:54.685998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:54.689643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.689643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.689643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.689643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:54.690075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:54.808901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1055/2103] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:54 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:54.810116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:54.820775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0614f518-ff3e-488b-a89c-b0b2ef7180eb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:54.825478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:54.826147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.826427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.826698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.827423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:54.829067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:54.843816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1049/2104] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:54.848977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:54.859506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16f4c-5fc0-49ce-9c34-0c216d8deb29 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:54.864831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0745-c1dc-4e3f-b488-16108af12362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:54.866011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0745-c1dc-4e3f-b488-16108af12362 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.866011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0745-c1dc-4e3f-b488-16108af12362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.866213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0745-c1dc-4e3f-b488-16108af12362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.866993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0745-c1dc-4e3f-b488-16108af12362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:54.868681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0745-c1dc-4e3f-b488-16108af12362 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:54.879373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1056/2105] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:28:54.883863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:54.894550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875802e-d497-4e30-a50c-bb90e4c91917 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:54.899305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:54.900251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:54.900549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:54.900830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] PATH_INFO: `/v3/projects/48823f48a90240018ca90fb3643c95ec/users/dc3b2cdf64604191a5b7c608271a9030/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:54.917280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48823f48a90240018ca90fb3643c95ec, user_id=dc3b2cdf64604191a5b7c608271a9030, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:54.920044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:55.027281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1050/2106] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:54 2022] PUT /identity/v3/projects/48823f48a90240018ca90fb3643c95ec/users/dc3b2cdf64604191a5b7c608271a9030/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:55.032441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:55.032731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:55.033015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:55.052251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:55.053725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:55.064343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:55.124044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1057/2107] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:55.131237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:55.133420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:55.203620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f08c4a-8630-4402-8ddf-b18323c29b37 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:55.208339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c2e368-2142-4193-9094-62785ad378e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:55.209079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2e368-2142-4193-9094-62785ad378e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:55.209382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2e368-2142-4193-9094-62785ad378e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:55.209777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c2e368-2142-4193-9094-62785ad378e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:55.222274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2e368-2142-4193-9094-62785ad378e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:55.224498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c2e368-2142-4193-9094-62785ad378e9 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:55.258745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1051/2108] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:55.778277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:55.779807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:55.791985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09201413-1adc-4664-b577-21e0bf1b97a5 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:55.796694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:55.797672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:55.797957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:55.798237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] PATH_INFO: `/v3/users/e8df94da49dd4619a4420905934af4c7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:55.799435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8df94da49dd4619a4420905934af4c7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:55.802017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:56.037756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8df94da49dd4619a4420905934af4c7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:56.052549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8df94da49dd4619a4420905934af4c7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:56.142250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8df94da49dd4619a4420905934af4c7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:56.142832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8df94da49dd4619a4420905934af4c7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:56.143422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8df94da49dd4619a4420905934af4c7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:56.157782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1058/2109] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:28:55 2022] DELETE /identity/v3/users/e8df94da49dd4619a4420905934af4c7 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:56.163461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711845b-9cce-4d66-b450-c62ac59908df None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:56.164005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711845b-9cce-4d66-b450-c62ac59908df None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:56.164099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711845b-9cce-4d66-b450-c62ac59908df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:56.183568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f711845b-9cce-4d66-b450-c62ac59908df None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:56.184877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f711845b-9cce-4d66-b450-c62ac59908df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:56.250702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1052/2110] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:56.260274 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:56.261730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:56.321187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fe809-e0dd-4d42-8021-8455557363dc admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:56.325086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:56.325735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:56.326012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:56.326290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:56.338361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:56.340020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:56.351393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1059/2111] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:56.716209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:56.716497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:56.716942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:56.743561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:56.746934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] MFA Rules not processed for user `dc3b2cdf64604191a5b7c608271a9030`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:56.748203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:56.794353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:56.795824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:56.810084 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7be095-3cbb-45e6-b0d8-a7548c64f34e service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:56.815316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:56.816445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:56.816922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:56.817384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] PATH_INFO: `/v3/projects/1ec4faf08b8a4cd6826a94616e61437a` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:56.818934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ec4faf08b8a4cd6826a94616e61437a)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:56.820710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1053/2112] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:56.821757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:56.824673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:56.826357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:56.842937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64038e4a-106b-48f2-81a1-30fc7b980f45 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:56.848348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:56.849017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:56.849304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:56.849587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:56.850279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:56.852245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:56.966613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:56.983956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ec4faf08b8a4cd6826a94616e61437a', 'request_id': 'req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:56.990554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ec4faf08b8a4cd6826a94616e61437a', 'request_id': 'req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:56.996766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ec4faf08b8a4cd6826a94616e61437a', 'request_id': 'req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:57.001895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1054/2113] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:56 2022] POST /identity/v3/projects => generated 371 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:57.006114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.071210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca397c6-b2fd-49e4-8aaa-2d8c3f578696 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.075902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.076577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.076856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.077131 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.077817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.079730 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.130860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] The token cache is being invalidate because project 1ec4faf08b8a4cd6826a94616e61437a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:28:57.138467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:57.138467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fbc78135-ea07-41fe-ae98-ff1df8e4cbe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:28:57.138467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1060/2114] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:28:56 2022] DELETE /identity/v3/projects/1ec4faf08b8a4cd6826a94616e61437a => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:57.139822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6060d8c-cd1a-4e09-a651-dbc68448938a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.140445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6060d8c-cd1a-4e09-a651-dbc68448938a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.140746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6060d8c-cd1a-4e09-a651-dbc68448938a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.161399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c6060d8c-cd1a-4e09-a651-dbc68448938a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:57.162632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c6060d8c-cd1a-4e09-a651-dbc68448938a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:57.170718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1061/2115] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:57.182574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efcd1b-9e29-4bd2-9ca0-4cc042419436 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.182897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efcd1b-9e29-4bd2-9ca0-4cc042419436 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.183168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efcd1b-9e29-4bd2-9ca0-4cc042419436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.201357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1055/2116] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:57 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:57.202162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e0efcd1b-9e29-4bd2-9ca0-4cc042419436 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:57.203428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e0efcd1b-9e29-4bd2-9ca0-4cc042419436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:57.207784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.209985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:57.214814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1062/2117] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:57.219237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efcd1b-9e29-4bd2-9ca0-4cc042419436 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.225544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a886dbc1-61fd-439a-855e-d6e0bdcc6a2a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.230514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.231642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efcd1b-9e29-4bd2-9ca0-4cc042419436 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.231812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.231812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.231812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.232487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.234099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.237155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.238064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.238438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.238811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.239659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:28:57.240120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.241862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.250512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1056/2118] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:57.251564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.257029 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1063/2119] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:28:57.261228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.262575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb960af5-23bb-4eb5-93c1-0bce58b205f6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.268016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.268910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.269734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.270010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] PATH_INFO: `/v3/projects/257136c5554842f0a9cdaa5b8f1a988b/users/8fbae861e3704e5e91a6625b514e45b8/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.281295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a4ad-069e-405e-be4d-d1e0b278eb22 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.286777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.287528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.288419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.288419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.289010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.292119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.300956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257136c5554842f0a9cdaa5b8f1a988b, user_id=8fbae861e3704e5e91a6625b514e45b8, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.305793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.382949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1064/2120] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:28:57 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:28:57.387054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.409731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1057/2121] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:57 2022] PUT /identity/v3/projects/257136c5554842f0a9cdaa5b8f1a988b/users/8fbae861e3704e5e91a6625b514e45b8/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:57.434862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.468455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5178ae-5cf5-4e56-8c71-5b7d6888ec3a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.483254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:57.509539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0be813e-9871-4132-9cb5-096c73c497e6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.514108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.515148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.515148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.515518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.516325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.518054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.529846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1058/2122] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:57.534426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.539847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.540631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.540998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.541349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.542314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.546280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff92d80-aceb-4117-9769-68c0b1ee78d5 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.549758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.552714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.554073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.554357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.554631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] PATH_INFO: `/v3/projects/257136c5554842f0a9cdaa5b8f1a988b/users/8fbae861e3704e5e91a6625b514e45b8/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.556912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257136c5554842f0a9cdaa5b8f1a988b, user_id=8fbae861e3704e5e91a6625b514e45b8, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.559931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.575205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1059/2123] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:57 2022] PUT /identity/v3/projects/257136c5554842f0a9cdaa5b8f1a988b/users/8fbae861e3704e5e91a6625b514e45b8/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:57.778260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1065/2124] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:28:57 2022] POST /identity/v3/users => generated 290 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:28:57.779479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.845285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd6f94-c85c-41b0-b180-70259cd673c6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.850219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.850894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.851182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.851459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.852291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.853933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.869435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1060/2125] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:57.873604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.876522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:57.882702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6285d4-73f6-4ca4-bd23-7325d5ea8fa4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.888067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb268ef-6826-479d-95d2-79491b3c8fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.888731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb268ef-6826-479d-95d2-79491b3c8fca admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.889013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb268ef-6826-479d-95d2-79491b3c8fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.889291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb268ef-6826-479d-95d2-79491b3c8fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.890013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb268ef-6826-479d-95d2-79491b3c8fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.891608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb268ef-6826-479d-95d2-79491b3c8fca admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.901520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1066/2126] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:28:57.905945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:57.915982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfab62a-e8a1-4af3-88b4-42221a924839 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:57.920611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:57.921490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:57.921775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:57.922048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] PATH_INFO: `/v3/projects/87d8cb3f18ae4f05a85dc21abaf316cb/users/88575c0bd9db4fa8bc9bd511c0ac37dd/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:57.937930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] RBAC: Authorizing `identity:create_grant(project_id=87d8cb3f18ae4f05a85dc21abaf316cb, user_id=88575c0bd9db4fa8bc9bd511c0ac37dd, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:57.942644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:57.999382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1061/2127] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:28:57 2022] PUT /identity/v3/projects/87d8cb3f18ae4f05a85dc21abaf316cb/users/88575c0bd9db4fa8bc9bd511c0ac37dd/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:28:58.005120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090cd5-66d3-44cd-88f8-cbc622f8ccc8 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:58.005727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090cd5-66d3-44cd-88f8-cbc622f8ccc8 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:58.006074 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50090cd5-66d3-44cd-88f8-cbc622f8ccc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:58.024253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-50090cd5-66d3-44cd-88f8-cbc622f8ccc8 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:58.025401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-50090cd5-66d3-44cd-88f8-cbc622f8ccc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:58.094949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1067/2128] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:28:58.104741 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:58.107776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:58.184719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44fdd9e-43f8-4be2-a0f0-f226b584223e admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:58.189406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:58.190111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:58.190399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:58.190711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:58.205343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:58.206868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:58.219338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1062/2129] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:28:59.321314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:59.321828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:59.321926 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:59.350985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] MFA Rules not processed for user `8fbae861e3704e5e91a6625b514e45b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:28:59.352905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:59.447900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1068/2130] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:28:59.462479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:28:59.464913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:59.510694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:28:59.542231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5394586-beb2-448a-82bd-4514acdd88b9 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:28:59.546938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ef9ba6-64a8-474a-8da2-01acec5d0823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:28:59.549268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef9ba6-64a8-474a-8da2-01acec5d0823 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:28:59.549268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef9ba6-64a8-474a-8da2-01acec5d0823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:28:59.549268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef9ba6-64a8-474a-8da2-01acec5d0823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:28:59.552126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b9ef9ba6-64a8-474a-8da2-01acec5d0823 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:28:59.617461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef9ba6-64a8-474a-8da2-01acec5d0823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:28:59.618988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ef9ba6-64a8-474a-8da2-01acec5d0823 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:28:59.628725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1063/2131] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:28:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:02.466493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:02.469031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:02.517034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467ded4-b418-4bb5-b09e-358297213091 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:02.523335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:02.524715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:02.525201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:02.525645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:02.540359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:02.542040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:02.563841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1069/2132] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:04.695718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9e944-cb37-4d2e-967c-2b89c00358c1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:04.696285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9e944-cb37-4d2e-967c-2b89c00358c1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:04.696371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f9e944-cb37-4d2e-967c-2b89c00358c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:04.732609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-54f9e944-cb37-4d2e-967c-2b89c00358c1 None None] MFA Rules not processed for user `88575c0bd9db4fa8bc9bd511c0ac37dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:04.733850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-54f9e944-cb37-4d2e-967c-2b89c00358c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:04.805836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1064/2133] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:04.810346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:04.821853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5e314-501c-4860-862a-5ae2c918f5e3 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:04.826334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:04.826998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:04.827287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:04.827572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:04.828304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:04.830242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:04.948655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1070/2134] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:04 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:04.952912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9e944-cb37-4d2e-967c-2b89c00358c1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:04.964172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f9e944-cb37-4d2e-967c-2b89c00358c1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:05.023019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:05.023704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:05.024196 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:05.024484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:05.025192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:05.027112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:05.104121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1065/2135] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:04 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:05.109105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:05.121951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af5764-02fe-4f38-9269-5e624c904ccf admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:05.135477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:05.136262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:05.136614 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:05.136958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:05.137755 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:05.139472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:05.150684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1071/2136] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:05.157343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:05.168062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2992b-b26f-46c7-bb99-2e9f22b80395 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:05.173244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:05.174291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:05.174644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:05.175598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] PATH_INFO: `/v3/projects/ddf52d1949a6436da70dc66673708cd6/users/0adc392546a54ca5beee09ef53cc6be2/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:05.178745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:05.195035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf52d1949a6436da70dc66673708cd6, user_id=0adc392546a54ca5beee09ef53cc6be2, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:05.196684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:05.319051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1066/2137] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:05 2022] PUT /identity/v3/projects/ddf52d1949a6436da70dc66673708cd6/users/0adc392546a54ca5beee09ef53cc6be2/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:05.323256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:05.333833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29833a9-6434-4c68-8de4-c38c5c362282 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:05.342736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:05.394938 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:05.395589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:05.395928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:05.401795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:05.401795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:05.401795 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:05.411642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1072/2138] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:05.415946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:05.427177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cb5b6-d75f-4835-a2a9-0c302239d554 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:05.432348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:05.433236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:05.433516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:05.433785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] PATH_INFO: `/v3/projects/ddf52d1949a6436da70dc66673708cd6/users/0adc392546a54ca5beee09ef53cc6be2/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:05.435733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddf52d1949a6436da70dc66673708cd6, user_id=0adc392546a54ca5beee09ef53cc6be2, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:05.438523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:05.449717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1067/2139] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:05 2022] PUT /identity/v3/projects/ddf52d1949a6436da70dc66673708cd6/users/0adc392546a54ca5beee09ef53cc6be2/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:12.330337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:12.336873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:12.355224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3958bac-4662-41ad-9e55-76ce6608d8ed admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:12.450089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:12.451488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:12.451789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:12.452372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] PATH_INFO: `/v3/users/9858a1dcd4754d8c99b86ccfd2c8a359` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:12.453936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9858a1dcd4754d8c99b86ccfd2c8a359)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:12.456645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:12.694906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9858a1dcd4754d8c99b86ccfd2c8a359'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:12.730618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:12.734449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9858a1dcd4754d8c99b86ccfd2c8a359'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:12.738055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:12.762897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61cf22f-5a59-433e-90b0-b92bc92e9181 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:12.768657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:12.773770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:12.773770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:12.773770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] PATH_INFO: `/v3/users/f9af937105114b3bb02a228f5d851a9d` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:12.773770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9af937105114b3bb02a228f5d851a9d)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:12.774544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:12.821501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9858a1dcd4754d8c99b86ccfd2c8a359'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:12.828837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9858a1dcd4754d8c99b86ccfd2c8a359'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:12.828837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4d95d20f-433c-4661-acd0-5c8a64db4ad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9858a1dcd4754d8c99b86ccfd2c8a359'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:12.845101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1073/2140] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:12 2022] DELETE /identity/v3/users/9858a1dcd4754d8c99b86ccfd2c8a359 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:12.847436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441a235-2b71-49bc-937d-77108c1b865c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:12.848124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441a235-2b71-49bc-937d-77108c1b865c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:12.848723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441a235-2b71-49bc-937d-77108c1b865c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:12.893162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3441a235-2b71-49bc-937d-77108c1b865c None None] MFA Rules not processed for user `0adc392546a54ca5beee09ef53cc6be2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:12.894610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3441a235-2b71-49bc-937d-77108c1b865c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:12.913330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:12.993632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1074/2141] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:12.996164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:12.996566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:12.996935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:13.023277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:13.024833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:13.025783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9af937105114b3bb02a228f5d851a9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:13.045092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9af937105114b3bb02a228f5d851a9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:13.092137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1075/2142] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:13.093055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:13.094424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:13.164150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9af937105114b3bb02a228f5d851a9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:13.164969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9af937105114b3bb02a228f5d851a9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:13.165610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9af937105114b3bb02a228f5d851a9d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:13.179182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1068/2143] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:12 2022] DELETE /identity/v3/users/f9af937105114b3bb02a228f5d851a9d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:13.182250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:13.182250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:13.225347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514abb58-ba2d-4491-8e49-75ee82828695 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:13.248852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b1d20-47ac-4684-b0a4-8f5ad7e0a34a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:13.253334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7ecf5-5e31-4e33-96d7-67c3c074beee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:13.253997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7ecf5-5e31-4e33-96d7-67c3c074beee service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:13.254281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7ecf5-5e31-4e33-96d7-67c3c074beee service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:13.254585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7ecf5-5e31-4e33-96d7-67c3c074beee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:13.256898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-31f7ecf5-5e31-4e33-96d7-67c3c074beee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:13.296166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:13.296977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:13.297351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:13.297636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:13.299786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:13.321089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:13.333412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7ecf5-5e31-4e33-96d7-67c3c074beee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:13.334861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7ecf5-5e31-4e33-96d7-67c3c074beee service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:13.345438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1069/2144] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:13.347868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:13.348146 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:13.348444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:13.371179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:13.371901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:13.372447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:13.373767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:13.380708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1070/2145] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:13.384799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1076/2146] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:13.396603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:13.398044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:13.412018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c05d9c-4303-43ce-9043-a4d95edb10c8 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:13.416208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:13.416835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:13.417117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:13.422646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:13.428170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:13.429966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:13.443788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1071/2147] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:13.828992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:14.076511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:14.077433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:14.092380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fec24f-2b53-49a0-8cb1-de18cf79e08d service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:14.097229 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:14.098226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:14.098578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:14.098921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] PATH_INFO: `/v3/projects/523b7629c65848ffbc71ef954b7f9ddc` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:14.100374 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] RBAC: Authorizing `identity:delete_project(project_id=523b7629c65848ffbc71ef954b7f9ddc)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:14.105296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:14.263400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523b7629c65848ffbc71ef954b7f9ddc', 'request_id': 'req-d72581f2-4918-4efd-97c4-113e38df029e'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.269627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523b7629c65848ffbc71ef954b7f9ddc', 'request_id': 'req-d72581f2-4918-4efd-97c4-113e38df029e'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.281078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523b7629c65848ffbc71ef954b7f9ddc', 'request_id': 'req-d72581f2-4918-4efd-97c4-113e38df029e'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.281542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:14.281542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:14.295304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a6289-2c2e-4e3b-adca-9627b89cd45a service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:14.367938 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] The token cache is being invalidate because project 523b7629c65848ffbc71ef954b7f9ddc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:14.370765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.370765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.370765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1077/2148] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:14 2022] DELETE /identity/v3/projects/523b7629c65848ffbc71ef954b7f9ddc => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:14.374779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:14.378147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:14.382318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:14.385192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:14.385555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:14.385898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] PATH_INFO: `/v3/projects/1baf236cb2314f9e9ba0cfbaafa1f961` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:14.388801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1baf236cb2314f9e9ba0cfbaafa1f961)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:14.391580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:14.396827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72581f2-4918-4efd-97c4-113e38df029e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:14.413717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:14.416960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:14.416960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:14.416960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] PATH_INFO: `/v3/users/1859545f62684fa29a103fa0d0ca69d9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:14.417966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1859545f62684fa29a103fa0d0ca69d9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:14.421943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:14.522760 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1baf236cb2314f9e9ba0cfbaafa1f961', 'request_id': 'req-df14e007-d675-4532-b3f1-420b77baf38d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.532195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1baf236cb2314f9e9ba0cfbaafa1f961', 'request_id': 'req-df14e007-d675-4532-b3f1-420b77baf38d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.543136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1baf236cb2314f9e9ba0cfbaafa1f961', 'request_id': 'req-df14e007-d675-4532-b3f1-420b77baf38d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.690966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] The token cache is being invalidate because project 1baf236cb2314f9e9ba0cfbaafa1f961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:14.691193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.691892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.696642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:14.702520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1072/2149] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:14 2022] DELETE /identity/v3/projects/1baf236cb2314f9e9ba0cfbaafa1f961 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:14.702520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:14.702520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:14.740573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1859545f62684fa29a103fa0d0ca69d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.769916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1859545f62684fa29a103fa0d0ca69d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.807643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14e007-d675-4532-b3f1-420b77baf38d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:14.812811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:14.813704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:14.814029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:14.814483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] PATH_INFO: `/v3/users/bfd91082e3f14f6182592d14a027284c` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:14.815931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd91082e3f14f6182592d14a027284c)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:14.818768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:14.823522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:14.851488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1859545f62684fa29a103fa0d0ca69d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.852496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1859545f62684fa29a103fa0d0ca69d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.853355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1859545f62684fa29a103fa0d0ca69d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:14.874582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1078/2150] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:14 2022] DELETE /identity/v3/users/1859545f62684fa29a103fa0d0ca69d9 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:15.080353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd91082e3f14f6182592d14a027284c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.100712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd91082e3f14f6182592d14a027284c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.199988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd91082e3f14f6182592d14a027284c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.200982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd91082e3f14f6182592d14a027284c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.201789 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd91082e3f14f6182592d14a027284c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.219252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1073/2151] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:14 2022] DELETE /identity/v3/users/bfd91082e3f14f6182592d14a027284c => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:15.551590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:15.552848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:15.691881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d5ec4-5281-442b-9b20-a7770f588c6b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:15.788298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:15.789528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:15.790016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:15.790473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] PATH_INFO: `/v3/projects/c0486ad03b12460b9a278a8b1e4087e4` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:15.792077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0486ad03b12460b9a278a8b1e4087e4)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:15.794281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:15.938976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0486ad03b12460b9a278a8b1e4087e4', 'request_id': 'req-06a71ba5-b36a-447d-970d-b9b258d62658'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.949144 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0486ad03b12460b9a278a8b1e4087e4', 'request_id': 'req-06a71ba5-b36a-447d-970d-b9b258d62658'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.955666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0486ad03b12460b9a278a8b1e4087e4', 'request_id': 'req-06a71ba5-b36a-447d-970d-b9b258d62658'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:15.975710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:15.977238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:15.988119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:15.997924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cca9ab-3c2b-4edb-a245-fda3f984254a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.054516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] The token cache is being invalidate because project c0486ad03b12460b9a278a8b1e4087e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:16.054811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:16.062586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-06a71ba5-b36a-447d-970d-b9b258d62658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:16.062586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1079/2152] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:15 2022] DELETE /identity/v3/projects/c0486ad03b12460b9a278a8b1e4087e4 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:16.063142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abe647-17a1-4228-94cb-b34092a347ea None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.064525 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abe647-17a1-4228-94cb-b34092a347ea None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.064815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abe647-17a1-4228-94cb-b34092a347ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.112770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.117560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.117560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.117560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] PATH_INFO: `/v3/projects/0413d26c493a46d6910ce20b4b63e251` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.117560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0413d26c493a46d6910ce20b4b63e251)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.119646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.124367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-06abe647-17a1-4228-94cb-b34092a347ea None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:16.126288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-06abe647-17a1-4228-94cb-b34092a347ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:16.149625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1080/2153] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:16.157618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9f21e-f2cc-4f8e-acfb-5c99764e023d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.157968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9f21e-f2cc-4f8e-acfb-5c99764e023d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.158267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f9f21e-f2cc-4f8e-acfb-5c99764e023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.201278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-30f9f21e-f2cc-4f8e-acfb-5c99764e023d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:16.204763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-30f9f21e-f2cc-4f8e-acfb-5c99764e023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:16.223242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1081/2154] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:16.231362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9f21e-f2cc-4f8e-acfb-5c99764e023d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.237424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0413d26c493a46d6910ce20b4b63e251', 'request_id': 'req-f031325c-cc9a-4f3a-a4b5-8c27c7948783'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:16.242296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f9f21e-f2cc-4f8e-acfb-5c99764e023d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.245815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0413d26c493a46d6910ce20b4b63e251', 'request_id': 'req-f031325c-cc9a-4f3a-a4b5-8c27c7948783'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:16.259983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0413d26c493a46d6910ce20b4b63e251', 'request_id': 'req-f031325c-cc9a-4f3a-a4b5-8c27c7948783'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:16.321679 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.322514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.322907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.323377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.324230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:16.324562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.326197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.339468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1082/2155] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:29:16.343166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.353779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80038bd7-3120-43f0-ba29-d7ff3a2f9671 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.358555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.359215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.359495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.359977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.360728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.363222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.375216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] The token cache is being invalidate because project 0413d26c493a46d6910ce20b4b63e251 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:16.377660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:16.377660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f031325c-cc9a-4f3a-a4b5-8c27c7948783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:16.378142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1074/2156] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:15 2022] DELETE /identity/v3/projects/0413d26c493a46d6910ce20b4b63e251 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:16.383129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ab926-a2de-4ef2-ad5b-8c079f4a90f8 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.383546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ab926-a2de-4ef2-ad5b-8c079f4a90f8 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.383945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ab926-a2de-4ef2-ad5b-8c079f4a90f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.406502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-540ab926-a2de-4ef2-ad5b-8c079f4a90f8 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:16.406870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-540ab926-a2de-4ef2-ad5b-8c079f4a90f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:16.437248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1075/2157] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:16.445120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.445870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.445870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.459667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:16.476104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:16.477561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:16.494801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1083/2158] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:16.495137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1076/2159] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:16.500791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.501849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.502209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:16.515376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6eef42-4b29-43d0-b851-2d553afb5371 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.580590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ef4de-520a-4e25-bb03-3c5656d8559e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.587141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.587911 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.588195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.588524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.589160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.589340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:16.589782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.590041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.590719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.591342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.591857 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.592229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.595334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.607885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1084/2160] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:29:16.612392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.645625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190ce62a-b51f-4b1f-97eb-75a8f2978a31 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.650977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.651660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.652589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.653625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.654493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.656787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.725857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1077/2161] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/users => generated 294 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:16.729909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.740364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae45b7-7e86-43d0-a8f6-4ca9494ecb1c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.745854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.746518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.746911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.747233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.748052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.749702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.764270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1078/2162] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:16.769918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.785602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06245480-c2be-4394-8ae3-984c11bc68d8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.790467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1085/2163] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/projects => generated 367 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:16.791635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.792450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.792799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.793142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.794787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.794918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.796810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.812327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1079/2164] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:16.823955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:16.848332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4801076f-9f64-4409-b702-810a32e14c35 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.886357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5166d6-385f-4487-83e7-9e8dca88c76d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:16.894752 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.895903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.896466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.896792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.897670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.900376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:16.947700 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f5880-6754-48bf-b8a8-bb8b9752ad26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:16.949137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5880-6754-48bf-b8a8-bb8b9752ad26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:16.949661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5880-6754-48bf-b8a8-bb8b9752ad26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:16.950084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f5880-6754-48bf-b8a8-bb8b9752ad26 admin admin] PATH_INFO: `/v3/projects/047e67653ab143f18ca53c1dd434ebc4/users/181913c618c94eb995372c250a703dc9/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:16.971216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f5880-6754-48bf-b8a8-bb8b9752ad26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=047e67653ab143f18ca53c1dd434ebc4, user_id=181913c618c94eb995372c250a703dc9, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:16.975206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f5880-6754-48bf-b8a8-bb8b9752ad26 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.004773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1086/2165] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:16 2022] POST /identity/v3/users => generated 284 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:17.013091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.025960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3f4d4-f71e-42bc-9aa6-ba758a4b9f1d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.033775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.034721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.035004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.035350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.036593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.038902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.054217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1087/2166] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:17.060776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.080229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1080/2167] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:16 2022] PUT /identity/v3/projects/047e67653ab143f18ca53c1dd434ebc4/users/181913c618c94eb995372c250a703dc9/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:17.088102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22727983-6658-4641-b455-d3ef90abf589 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.096314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0623e-55ba-4fb2-8a7b-d78a61c5e1c5 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.096899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0623e-55ba-4fb2-8a7b-d78a61c5e1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.097361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0623e-55ba-4fb2-8a7b-d78a61c5e1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.152832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bdb0623e-55ba-4fb2-8a7b-d78a61c5e1c5 None None] MFA Rules not processed for user `181913c618c94eb995372c250a703dc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:17.154185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bdb0623e-55ba-4fb2-8a7b-d78a61c5e1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:17.230868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.231768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.234645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.234645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.234645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.243694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1081/2168] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:17.244745 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.244979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0623e-55ba-4fb2-8a7b-d78a61c5e1c5 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.269582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0623e-55ba-4fb2-8a7b-d78a61c5e1c5 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.273025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.273844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.273990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.274271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.274972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.276937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.280991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1088/2169] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:17.284861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.300633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d0893-a95b-4d20-a9b7-ef513fa4bbf9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.313838 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f1fa3-a636-48fb-a4cf-cbefb91ac592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.314754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1fa3-a636-48fb-a4cf-cbefb91ac592 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.315040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1fa3-a636-48fb-a4cf-cbefb91ac592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.315321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f1fa3-a636-48fb-a4cf-cbefb91ac592 admin admin] PATH_INFO: `/v3/projects/bc95968cd4b94040aa5abcb61e0039af/users/cdf7e11012334cae8099d4fd51fb5aec/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.323295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:17.335454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f1fa3-a636-48fb-a4cf-cbefb91ac592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc95968cd4b94040aa5abcb61e0039af, user_id=cdf7e11012334cae8099d4fd51fb5aec, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.338305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f1fa3-a636-48fb-a4cf-cbefb91ac592 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.379563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1082/2170] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:17 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:17.383533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.439170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1089/2171] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:17 2022] PUT /identity/v3/projects/bc95968cd4b94040aa5abcb61e0039af/users/cdf7e11012334cae8099d4fd51fb5aec/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:17.441349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59071-f6b1-4107-a1b4-d535f0c66f3f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.441634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59071-f6b1-4107-a1b4-d535f0c66f3f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.447363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a59071-f6b1-4107-a1b4-d535f0c66f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.459363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe90afc-ff35-475e-bbad-089cc566dd35 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.504079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-86a59071-f6b1-4107-a1b4-d535f0c66f3f None None] MFA Rules not processed for user `cdf7e11012334cae8099d4fd51fb5aec`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:17.506026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-86a59071-f6b1-4107-a1b4-d535f0c66f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:17.558617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.559944 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.560404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.560848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.561766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.565135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.653322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1090/2172] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:17.653322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59071-f6b1-4107-a1b4-d535f0c66f3f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.676018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a59071-f6b1-4107-a1b4-d535f0c66f3f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.689794 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1083/2173] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:17 2022] POST /identity/v3/users => generated 299 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:17.689983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.689983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.689983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.689983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.690367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.691192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.693210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.718525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549b9ae-bc56-468a-909b-a7f88838b577 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.733869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.736163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.736808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.737094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.738403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.740113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.747709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:17.758777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1084/2174] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:17.762865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.773754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec98e20-d869-4d4a-8aa6-456e72bcafff admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:17.782062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:17.783181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:17.783472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:17.784688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] PATH_INFO: `/v3/projects/5a47a16ba5f240adad4589c53e0c5070/users/4558173288a34356868a155b1effe683/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:17.814971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a47a16ba5f240adad4589c53e0c5070, user_id=4558173288a34356868a155b1effe683, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:17.817776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:17.831505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1091/2175] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:17 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:17.836062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.920529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1085/2176] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:17 2022] PUT /identity/v3/projects/5a47a16ba5f240adad4589c53e0c5070/users/4558173288a34356868a155b1effe683/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:17.924162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:17.932817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76315390-e280-4cf7-96aa-03085915872d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.001241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d888dcb-cbc7-497e-9d34-f510047eb1af admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.004123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.005061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.005463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.005747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.006535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.006678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.007343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.008294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.008651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.009226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.009389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.011041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.027653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1086/2177] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:18.028885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.041553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e70372-2140-44c0-8c0c-9657e35cf04f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.046627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ac40f-6ef6-403f-a85c-dff8ae78b441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.047802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ac40f-6ef6-403f-a85c-dff8ae78b441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.052969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ac40f-6ef6-403f-a85c-dff8ae78b441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.052969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ac40f-6ef6-403f-a85c-dff8ae78b441 admin admin] PATH_INFO: `/v3/projects/5a47a16ba5f240adad4589c53e0c5070/users/4558173288a34356868a155b1effe683/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.052969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ac40f-6ef6-403f-a85c-dff8ae78b441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a47a16ba5f240adad4589c53e0c5070, user_id=4558173288a34356868a155b1effe683, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.053512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ac40f-6ef6-403f-a85c-dff8ae78b441 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.063931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1087/2178] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:18 2022] PUT /identity/v3/projects/5a47a16ba5f240adad4589c53e0c5070/users/4558173288a34356868a155b1effe683/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:18.073883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.074173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.074449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.103133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] MFA Rules not processed for user `4558173288a34356868a155b1effe683`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:18.104394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:18.163835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1092/2179] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:17 2022] POST /identity/v3/users => generated 291 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:18.168147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.187936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffaf51e-f6d7-423b-b310-211767fcd4be admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.206583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1088/2180] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:18.270371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:18.272092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.272854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.273207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.273561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.274629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.276537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.299986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.304051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:18.309817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1093/2181] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:18.311613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.327888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c664d34-eb72-4d0a-bf06-2e33b980e9ad admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.333104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.334432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.334869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.335424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] PATH_INFO: `/v3/projects/6bfd3187745f41268e9dd44287865b24/users/7ec12042870c4093b1338d9b0ef40354/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.353307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfd3187745f41268e9dd44287865b24, user_id=7ec12042870c4093b1338d9b0ef40354, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.358856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.380876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbadac71-46c1-44a5-ad53-6df2072f5968 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.384584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.385384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.385668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.385941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.395872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.397388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.407036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1089/2182] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:18.443076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.457560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358be7b7-7229-43dd-a7de-297665757768 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.465345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.465345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.465345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.465345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] PATH_INFO: `/v3/users/181913c618c94eb995372c250a703dc9` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.466260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=181913c618c94eb995372c250a703dc9)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.468485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.486678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1094/2183] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:18 2022] PUT /identity/v3/projects/6bfd3187745f41268e9dd44287865b24/users/7ec12042870c4093b1338d9b0ef40354/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:18.490781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.596275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef71e7-245c-4dd3-af0a-36bfe71d383e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.602001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.602947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.603415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.603901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.604915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.607201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.622162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1095/2184] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:18 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:18.623724 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.635481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0b4ce-9f9d-4662-84af-3bea509c3ac4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.641156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23ddfd-3a6c-40d4-a02a-f956c874b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.642122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23ddfd-3a6c-40d4-a02a-f956c874b2f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.642469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23ddfd-3a6c-40d4-a02a-f956c874b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.642849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23ddfd-3a6c-40d4-a02a-f956c874b2f9 admin admin] PATH_INFO: `/v3/projects/6bfd3187745f41268e9dd44287865b24/users/7ec12042870c4093b1338d9b0ef40354/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.645610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23ddfd-3a6c-40d4-a02a-f956c874b2f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfd3187745f41268e9dd44287865b24, user_id=7ec12042870c4093b1338d9b0ef40354, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.649023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23ddfd-3a6c-40d4-a02a-f956c874b2f9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.662072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:18.666337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1096/2185] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:18 2022] PUT /identity/v3/projects/6bfd3187745f41268e9dd44287865b24/users/7ec12042870c4093b1338d9b0ef40354/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:18.678072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.678629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.678913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.709597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] MFA Rules not processed for user `7ec12042870c4093b1338d9b0ef40354`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:18.710941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:18.797889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1097/2186] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:18 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:18.808023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181913c618c94eb995372c250a703dc9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:18.810335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:18.811872 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:18.825623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181913c618c94eb995372c250a703dc9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:18.880092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbfaea-47f1-45ac-88b4-0f6e7490a8b6 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:18.884058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ec3e0-ab22-4db9-92d8-70b2a7301661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:18.884714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ec3e0-ab22-4db9-92d8-70b2a7301661 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.885110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ec3e0-ab22-4db9-92d8-70b2a7301661 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.885439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ec3e0-ab22-4db9-92d8-70b2a7301661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.895365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ec3e0-ab22-4db9-92d8-70b2a7301661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:18.896217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181913c618c94eb995372c250a703dc9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:18.896965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ec3e0-ab22-4db9-92d8-70b2a7301661 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:18.897140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181913c618c94eb995372c250a703dc9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:18.897826 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181913c618c94eb995372c250a703dc9'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:18.908021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1098/2187] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:18 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:18.914515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1090/2188] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:18 2022] DELETE /identity/v3/users/181913c618c94eb995372c250a703dc9 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:18.917421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:18.917788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:18.918134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:18.938839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:18.940176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:19.010681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1099/2189] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:19.022830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:19.028708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:19.034611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:19.037044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:19.110528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f82b1c-abcc-491d-9340-4c1dc87d4735 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:19.116578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48840d1-4621-4b44-8641-50a69e0d447d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:19.117854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:19.119052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:19.119581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:19.120223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:19.121122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:19.121792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:19.122076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:19.122525 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:19.125006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:19.132867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:19.134751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:19.160255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1091/2190] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:19.200258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:19.202959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:19.204499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:19.228008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1100/2191] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:19.325074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:19.662042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:19.858047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:19.859470 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:19.872878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d5020-6481-48e6-a1f9-64692afab1fb service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:19.877652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:19.879111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:19.879383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:19.879660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] PATH_INFO: `/v3/projects/047e67653ab143f18ca53c1dd434ebc4` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:19.881020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=047e67653ab143f18ca53c1dd434ebc4)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:19.882952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:20.053259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '047e67653ab143f18ca53c1dd434ebc4', 'request_id': 'req-4483cd35-e42a-469c-8701-82c81dae05e3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.060177 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '047e67653ab143f18ca53c1dd434ebc4', 'request_id': 'req-4483cd35-e42a-469c-8701-82c81dae05e3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.067784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '047e67653ab143f18ca53c1dd434ebc4', 'request_id': 'req-4483cd35-e42a-469c-8701-82c81dae05e3'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.211555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] The token cache is being invalidate because project 047e67653ab143f18ca53c1dd434ebc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:20.212223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.213648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.215539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1092/2192] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:19 2022] DELETE /identity/v3/projects/047e67653ab143f18ca53c1dd434ebc4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:20.219285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:20.231392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dddb1d-971c-441b-9de3-79e89c75366b service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:20.289053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:20.289931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:20.290274 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:20.290635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] PATH_INFO: `/v3/users/4558173288a34356868a155b1effe683` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:20.292004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4558173288a34356868a155b1effe683)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:20.293898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:20.712338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4558173288a34356868a155b1effe683'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.731273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4558173288a34356868a155b1effe683'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.855200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:20.907018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4558173288a34356868a155b1effe683'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.907695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4558173288a34356868a155b1effe683'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.908358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7f65db75-931e-45b1-aa92-41a0bce67ae2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4558173288a34356868a155b1effe683'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:20.926191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1101/2193] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:20 2022] DELETE /identity/v3/users/4558173288a34356868a155b1effe683 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:21.491049 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:21.492492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:21.519941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483cd35-e42a-469c-8701-82c81dae05e3 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:21.558261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:21.591546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:21.592539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:21.592909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:21.593232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] PATH_INFO: `/v3/projects/5a47a16ba5f240adad4589c53e0c5070` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:21.594444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a47a16ba5f240adad4589c53e0c5070)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:21.596469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:21.747416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a47a16ba5f240adad4589c53e0c5070', 'request_id': 'req-cff9fb54-6366-4e87-9a92-c568defe4ca0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:21.754998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a47a16ba5f240adad4589c53e0c5070', 'request_id': 'req-cff9fb54-6366-4e87-9a92-c568defe4ca0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:21.760682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a47a16ba5f240adad4589c53e0c5070', 'request_id': 'req-cff9fb54-6366-4e87-9a92-c568defe4ca0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:21.844605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] The token cache is being invalidate because project 5a47a16ba5f240adad4589c53e0c5070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:21.844605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:21.845949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cff9fb54-6366-4e87-9a92-c568defe4ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:21.848414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1093/2194] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:21 2022] DELETE /identity/v3/projects/5a47a16ba5f240adad4589c53e0c5070 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:21.858307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63915551-875f-4ab9-a9f7-70db8353c256 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:21.858735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63915551-875f-4ab9-a9f7-70db8353c256 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:21.859331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63915551-875f-4ab9-a9f7-70db8353c256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:21.878242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-63915551-875f-4ab9-a9f7-70db8353c256 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:21.879635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-63915551-875f-4ab9-a9f7-70db8353c256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:21.952275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1102/2195] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:21.960647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100db0d3-a8c5-4b41-a71e-f0be261050ab None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:21.961006 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100db0d3-a8c5-4b41-a71e-f0be261050ab None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:21.961274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100db0d3-a8c5-4b41-a71e-f0be261050ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:21.980859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-100db0d3-a8c5-4b41-a71e-f0be261050ab None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:21.982096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-100db0d3-a8c5-4b41-a71e-f0be261050ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:21.990564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1094/2196] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:21.995002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63915551-875f-4ab9-a9f7-70db8353c256 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.006494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63915551-875f-4ab9-a9f7-70db8353c256 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.011500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.012219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.012570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.012810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.013540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:22.013850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.015473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:22.028214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1103/2197] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:29:22.033035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100db0d3-a8c5-4b41-a71e-f0be261050ab None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.043232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100db0d3-a8c5-4b41-a71e-f0be261050ab None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.048703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.049369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.049654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.049957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.050791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.053628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:22.171758 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1095/2198] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:22 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:22.175982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.186548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212135d-690f-48db-a5bd-763b199b1ab0 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.255180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.255792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.256079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.256355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.257062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.260279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:22.268766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:22.349045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1104/2199] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:22 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:22.353086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.363545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d8138-1eb1-4fe8-8913-0833e069002a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.368887 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.369581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.369870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.370147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.370988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.373066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:22.386622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1096/2200] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:22.390637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.406340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13743fba-53c3-4200-a146-8cffd3df2f04 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.411362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6f617-e767-472d-83c8-c6c18033ebe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.412062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f617-e767-472d-83c8-c6c18033ebe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.412349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f617-e767-472d-83c8-c6c18033ebe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.412627 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6f617-e767-472d-83c8-c6c18033ebe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.413352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6f617-e767-472d-83c8-c6c18033ebe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.415021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6f617-e767-472d-83c8-c6c18033ebe7 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:22.431780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1105/2201] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:22.440326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.446292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc989a3-38c2-449f-9fb1-1c4ea776dcaf admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.451640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.452631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.452913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.453189 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] PATH_INFO: `/v3/projects/dc5e828b9b8f44c7b578480966ea4cd2/users/1727f0c55a1d46c2b693e3ea8bfada9a/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.475145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5e828b9b8f44c7b578480966ea4cd2, user_id=1727f0c55a1d46c2b693e3ea8bfada9a, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.478163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:22.565210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1097/2202] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:22 2022] PUT /identity/v3/projects/dc5e828b9b8f44c7b578480966ea4cd2/users/1727f0c55a1d46c2b693e3ea8bfada9a/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:22.579551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2406-ca42-4a26-ad2a-fccfb6e3493d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.580330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2406-ca42-4a26-ad2a-fccfb6e3493d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.580621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df2406-ca42-4a26-ad2a-fccfb6e3493d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.610549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-96df2406-ca42-4a26-ad2a-fccfb6e3493d None None] MFA Rules not processed for user `1727f0c55a1d46c2b693e3ea8bfada9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:22.611785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-96df2406-ca42-4a26-ad2a-fccfb6e3493d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:22.680285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1106/2203] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:22.684910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.756247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:22.766575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846607b3-6a6d-4e6e-9142-535ea6520743 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.771314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.772001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.772283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.772559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.773254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.775187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:22.895302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1098/2204] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:22 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:22.896690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2406-ca42-4a26-ad2a-fccfb6e3493d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:22.908181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df2406-ca42-4a26-ad2a-fccfb6e3493d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:22.967016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:22.967692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:22.968012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:22.968335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:22.969037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:22.970960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.056027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1107/2205] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:22 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:23.059882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.070583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b2bbc-00d1-486d-bf3b-72aa53216ef7 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.075155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.075970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.076255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.076528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.077246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.078821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.092937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1099/2206] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:23.093515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.107675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4d8f7-a2db-4c39-88da-9201c9117bd1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.114118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c353-02a7-4c71-8f27-adea488c00f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.114787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c353-02a7-4c71-8f27-adea488c00f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.115066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c353-02a7-4c71-8f27-adea488c00f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.115344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c353-02a7-4c71-8f27-adea488c00f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.120966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c353-02a7-4c71-8f27-adea488c00f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.120966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243c353-02a7-4c71-8f27-adea488c00f9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.131177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1108/2207] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:23.132551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.142917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e2576-cb44-4815-8e54-1d085a31ea5d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.148201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.149107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.149385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.149661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] PATH_INFO: `/v3/projects/c91c9e9b15ab4a0889e15494ae781ca7/users/e9f7bf9e901f4ab8a8032f5d597caa2a/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.165775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91c9e9b15ab4a0889e15494ae781ca7, user_id=e9f7bf9e901f4ab8a8032f5d597caa2a, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.168453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.239674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1100/2208] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:23 2022] PUT /identity/v3/projects/c91c9e9b15ab4a0889e15494ae781ca7/users/e9f7bf9e901f4ab8a8032f5d597caa2a/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:23.249850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2397d63-0690-4144-bf3c-efc432c80ade None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.250138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2397d63-0690-4144-bf3c-efc432c80ade None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.250414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2397d63-0690-4144-bf3c-efc432c80ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.279291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c2397d63-0690-4144-bf3c-efc432c80ade None None] MFA Rules not processed for user `e9f7bf9e901f4ab8a8032f5d597caa2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:23.280566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c2397d63-0690-4144-bf3c-efc432c80ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:23.324291 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:23.351391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1109/2209] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:23.356299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.426658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d99151-a80a-494c-aaf5-cef44ebd18e2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.433027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.433850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.434025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.434286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.434998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.437359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.544208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1101/2210] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:23 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:23.548502 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2397d63-0690-4144-bf3c-efc432c80ade None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.560157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2397d63-0690-4144-bf3c-efc432c80ade None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.619205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.619943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.620230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.624955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.624955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.624955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.713767 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1110/2211] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:23 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:23.717914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.729199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25c3f72-57fc-4305-babd-c14b01e3238b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.734065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.734754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.735033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.735320 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.736163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.738504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.753243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1102/2212] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:23.756423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.774532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8234e81d-6d88-455f-8d36-0c2c0a92614d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.779278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.780196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.780476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.780750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] PATH_INFO: `/v3/projects/5fb0020ce57743cab1ec0e4efc27d7e9/users/2c7d1d0513334de398d6c8fccd87dfb2/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.796526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb0020ce57743cab1ec0e4efc27d7e9, user_id=2c7d1d0513334de398d6c8fccd87dfb2, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.799256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.880605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1111/2213] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:23 2022] PUT /identity/v3/projects/5fb0020ce57743cab1ec0e4efc27d7e9/users/2c7d1d0513334de398d6c8fccd87dfb2/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:23.884525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.896837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214aefb-0048-4d01-a973-a5939119e3eb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:23.964104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1960514-af2a-479b-9ea9-b901769a471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:23.964798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1960514-af2a-479b-9ea9-b901769a471f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:23.965103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1960514-af2a-479b-9ea9-b901769a471f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:23.965364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1960514-af2a-479b-9ea9-b901769a471f admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:23.966104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1960514-af2a-479b-9ea9-b901769a471f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:23.967983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1960514-af2a-479b-9ea9-b901769a471f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:23.984188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1103/2214] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:23.987504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:23.998195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec410599-e217-47de-9930-69e1e905ecf6 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:24.002888 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:24.003834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:24.004118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:24.004395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] PATH_INFO: `/v3/projects/5fb0020ce57743cab1ec0e4efc27d7e9/users/2c7d1d0513334de398d6c8fccd87dfb2/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:24.006376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fb0020ce57743cab1ec0e4efc27d7e9, user_id=2c7d1d0513334de398d6c8fccd87dfb2, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:24.009103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:24.020774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1112/2215] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:23 2022] PUT /identity/v3/projects/5fb0020ce57743cab1ec0e4efc27d7e9/users/2c7d1d0513334de398d6c8fccd87dfb2/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:24.032018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:24.032328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:24.032581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:24.057126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:24.060057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] MFA Rules not processed for user `2c7d1d0513334de398d6c8fccd87dfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:24.061276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:24.136324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1104/2216] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:24.147709 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:24.149544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:24.208914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a4d49-5952-41d4-b830-087e578f4ec0 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:24.212965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5362674c-7d9d-42ef-93c3-950b09548828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:24.213612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362674c-7d9d-42ef-93c3-950b09548828 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:24.214096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362674c-7d9d-42ef-93c3-950b09548828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:24.214235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5362674c-7d9d-42ef-93c3-950b09548828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:24.224902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362674c-7d9d-42ef-93c3-950b09548828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:24.226696 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5362674c-7d9d-42ef-93c3-950b09548828 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:24.236991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1113/2217] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:24.790869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:24.792375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:24.872866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5d4572-ec4a-4b37-9c40-5e91a7535959 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:24.877539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:24.878445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:24.878731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:24.879011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] PATH_INFO: `/v3/users/cdf7e11012334cae8099d4fd51fb5aec` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:24.880253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf7e11012334cae8099d4fd51fb5aec)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:24.882167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:25.169632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf7e11012334cae8099d4fd51fb5aec'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:25.186856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf7e11012334cae8099d4fd51fb5aec'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:25.306795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf7e11012334cae8099d4fd51fb5aec'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:25.307455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf7e11012334cae8099d4fd51fb5aec'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:25.308291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf7e11012334cae8099d4fd51fb5aec'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:25.330070 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:25.330593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:25.330796 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:25.331824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1105/2218] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:24 2022] DELETE /identity/v3/users/cdf7e11012334cae8099d4fd51fb5aec => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:25.349481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:25.350693 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:25.389450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:25.439531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1114/2219] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:25.450499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:25.452463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:25.476379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:25.521737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7951bb-e36b-4b58-be2b-538b3d4b568c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:25.530563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:25.536086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:25.536639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:25.537997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:25.549066 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:25.551133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:25.566292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1106/2220] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:25.599476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:25.601033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:25.683935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09e8da-b072-42db-816d-df9954ae86d0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:25.690338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:25.691009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:25.691295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:25.691573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:25.694012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:25.759854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:25.762351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:25.773756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1115/2221] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:26.092494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:26.094029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:26.109372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a317ab-1f82-4c47-b634-3b78b09bcba6 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:26.115951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:26.116746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:26.117158 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:26.117274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] PATH_INFO: `/v3/projects/bc95968cd4b94040aa5abcb61e0039af` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:26.119451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc95968cd4b94040aa5abcb61e0039af)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:26.121807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:26.265341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc95968cd4b94040aa5abcb61e0039af', 'request_id': 'req-78644461-9f15-4b29-8307-41d081ecdebf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:26.272680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc95968cd4b94040aa5abcb61e0039af', 'request_id': 'req-78644461-9f15-4b29-8307-41d081ecdebf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:26.278422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc95968cd4b94040aa5abcb61e0039af', 'request_id': 'req-78644461-9f15-4b29-8307-41d081ecdebf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:26.380816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] The token cache is being invalidate because project bc95968cd4b94040aa5abcb61e0039af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:26.381127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:26.381759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:26.383312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1107/2222] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:26 2022] DELETE /identity/v3/projects/bc95968cd4b94040aa5abcb61e0039af => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:26.387339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:26.398617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5468b-b69e-4469-be3e-550ffba53a46 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:26.460780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:26.461432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:26.461808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:26.462091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] PATH_INFO: `/v3/users/7ec12042870c4093b1338d9b0ef40354` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:26.463295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ec12042870c4093b1338d9b0ef40354)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:26.465316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:26.845998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:26.859124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ec12042870c4093b1338d9b0ef40354'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:26.873449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ec12042870c4093b1338d9b0ef40354'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.078325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ec12042870c4093b1338d9b0ef40354'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.078936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ec12042870c4093b1338d9b0ef40354'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.079564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a2fbb0c9-4bff-47c8-94a4-44b4192565a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ec12042870c4093b1338d9b0ef40354'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.092692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1116/2223] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:26 2022] DELETE /identity/v3/users/7ec12042870c4093b1338d9b0ef40354 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:27.550222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:27.551782 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:27.580263 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78644461-9f15-4b29-8307-41d081ecdebf admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:27.656211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:27.657132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:27.657710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:27.657710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] PATH_INFO: `/v3/projects/6bfd3187745f41268e9dd44287865b24` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:27.660739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfd3187745f41268e9dd44287865b24)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:27.662785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:27.811022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:27.814082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfd3187745f41268e9dd44287865b24', 'request_id': 'req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.821201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfd3187745f41268e9dd44287865b24', 'request_id': 'req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.832063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfd3187745f41268e9dd44287865b24', 'request_id': 'req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.906952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] The token cache is being invalidate because project 6bfd3187745f41268e9dd44287865b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:27.907255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.908088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8e15e083-0af0-49a2-ad77-9ea6cd0529bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:27.910293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1108/2224] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:27 2022] DELETE /identity/v3/projects/6bfd3187745f41268e9dd44287865b24 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:27.915201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dec06-4460-4720-b490-b626810bd8b9 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:27.915715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dec06-4460-4720-b490-b626810bd8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:27.915834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642dec06-4460-4720-b490-b626810bd8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:27.936020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-642dec06-4460-4720-b490-b626810bd8b9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:27.937339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-642dec06-4460-4720-b490-b626810bd8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:28.044517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1117/2225] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:28.050735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707239-0a57-4c4c-9286-794eacbbf109 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:28.051099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707239-0a57-4c4c-9286-794eacbbf109 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:28.051475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36707239-0a57-4c4c-9286-794eacbbf109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:28.073397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-36707239-0a57-4c4c-9286-794eacbbf109 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:28.076716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-36707239-0a57-4c4c-9286-794eacbbf109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:28.087809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1109/2226] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:28.090901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dec06-4460-4720-b490-b626810bd8b9 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:28.111607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642dec06-4460-4720-b490-b626810bd8b9 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:28.117489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:28.118072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:28.121823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:28.121823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:28.121823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:28.121823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:28.121823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:28.137067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1118/2227] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:29:28.142736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36707239-0a57-4c4c-9286-794eacbbf109 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:28.153680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36707239-0a57-4c4c-9286-794eacbbf109 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:28.158917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:28.159583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:28.159967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:28.160252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:28.160959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:28.163766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:28.428954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1110/2228] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:28 2022] POST /identity/v3/projects => generated 383 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:28.469025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:28.481253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2bfdb4-0fd4-4168-9337-e1bb60141de1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:28.540751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:28.541426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:28.541712 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:28.541988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:28.542678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:28.544628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:28.852586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1119/2229] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:28 2022] POST /identity/v3/users => generated 292 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:28.856973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:28.869809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91754947-f0bb-48b7-b6be-b950a758854d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:28.877771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:28.878856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:28.879168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:28.879445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:28.880227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:28.882516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:28.895493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1111/2230] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:28.900294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:28.910314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5f8ba-1a67-4626-a282-3ab77bc2ad41 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:28.915087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04ea73-bd3f-457f-94cc-41783c6420aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:28.915778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ea73-bd3f-457f-94cc-41783c6420aa admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:28.916364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ea73-bd3f-457f-94cc-41783c6420aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:28.916364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04ea73-bd3f-457f-94cc-41783c6420aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:28.917166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04ea73-bd3f-457f-94cc-41783c6420aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:28.918772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04ea73-bd3f-457f-94cc-41783c6420aa admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:28.929609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1120/2231] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:28.933719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:28.944983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e952bb-136a-429e-a0ad-442471fc6ff8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:28.949953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:28.950614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:28.950900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:28.951176 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] PATH_INFO: `/v3/projects/1605a75588584804bcf33e370aee7a6f/users/4b8ec2e7df9046d3a15644b7373302e5/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:28.967069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1605a75588584804bcf33e370aee7a6f, user_id=4b8ec2e7df9046d3a15644b7373302e5, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:28.969910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:29.085174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1112/2232] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:28 2022] PUT /identity/v3/projects/1605a75588584804bcf33e370aee7a6f/users/4b8ec2e7df9046d3a15644b7373302e5/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:29.095417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:29.095715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:29.096066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:29.125393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] MFA Rules not processed for user `4b8ec2e7df9046d3a15644b7373302e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:29.126605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:29.131385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:29.194356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1121/2233] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:29.198957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:29.228134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:29.230392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:29.267381 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871df80b-b310-4bca-b343-1d077e08288d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:29.273156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:29.273989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:29.274361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:29.274678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:29.275359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:29.277967 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:29.315393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349608d-bed4-4483-b816-ece99444e9cf None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:29.320945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:29.321831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:29.322099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:29.322472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] PATH_INFO: `/v3/users/dc3b2cdf64604191a5b7c608271a9030` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:29.325011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3b2cdf64604191a5b7c608271a9030)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:29.327093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:29.369885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1113/2234] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:29 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:29.371069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:29.431105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:29.437129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8d5de2-fbc2-4e6b-b116-2386a419a445 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:29.442644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:29.444295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:29.444581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:29.444859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:29.445576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:29.447665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:29.539368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3b2cdf64604191a5b7c608271a9030'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:29.551840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3b2cdf64604191a5b7c608271a9030'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:29.684817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3b2cdf64604191a5b7c608271a9030'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:29.689036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3b2cdf64604191a5b7c608271a9030'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:29.689036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3b2cdf64604191a5b7c608271a9030'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:29.702400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1122/2235] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:29 2022] DELETE /identity/v3/users/dc3b2cdf64604191a5b7c608271a9030 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:29.726707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1114/2236] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:29 2022] POST /identity/v3/users => generated 298 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:29.732065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:29.733599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:29.764383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:29.805488 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e86b2b-3baf-4116-9baa-9f2acfa091ee admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:29.810293 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:29.810964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:29.811245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:29.811941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:29.812289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:29.813912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:29.827104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1123/2237] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:29.830624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:29.838958 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb3cf5-3460-4ace-b1b0-aa71efe4a680 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:29.843657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:29.844586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:29.844866 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:29.845172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] PATH_INFO: `/v3/projects/376e09b38dcb4e619aa92dd021a10e13/users/a963ee0651124a5bad2c308cac2d210e/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:29.868139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376e09b38dcb4e619aa92dd021a10e13, user_id=a963ee0651124a5bad2c308cac2d210e, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:29.871117 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:29.971104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:29.973098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1115/2238] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:29 2022] PUT /identity/v3/projects/376e09b38dcb4e619aa92dd021a10e13/users/a963ee0651124a5bad2c308cac2d210e/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:29.973693 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:29.985622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:29.995968 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a074b7e-12e8-46ad-b9a0-a0cbcdf365ab admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:30.043501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c61fdd-b910-4ae3-92f8-17ec0140f614 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:30.049756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:30.050489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:30.050776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:30.051057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:30.051880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:30.058326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:30.079852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:30.080738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:30.081027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:30.081556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] PATH_INFO: `/v3/projects/48823f48a90240018ca90fb3643c95ec` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:30.083008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48823f48a90240018ca90fb3643c95ec)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:30.087874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1116/2239] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:30.088014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:30.092728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:30.104121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4bda-817b-4e4d-86f2-13f1648c9832 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:30.109411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32fe222-c43b-4c07-a275-4ba636effd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:30.110939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe222-c43b-4c07-a275-4ba636effd75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:30.111308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe222-c43b-4c07-a275-4ba636effd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:30.111653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32fe222-c43b-4c07-a275-4ba636effd75 admin admin] PATH_INFO: `/v3/projects/376e09b38dcb4e619aa92dd021a10e13/users/a963ee0651124a5bad2c308cac2d210e/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:30.113996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe222-c43b-4c07-a275-4ba636effd75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=376e09b38dcb4e619aa92dd021a10e13, user_id=a963ee0651124a5bad2c308cac2d210e, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:30.116905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32fe222-c43b-4c07-a275-4ba636effd75 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:30.130835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1117/2240] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:30 2022] PUT /identity/v3/projects/376e09b38dcb4e619aa92dd021a10e13/users/a963ee0651124a5bad2c308cac2d210e/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:30.143023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:30.144125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:30.144471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:30.173624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] MFA Rules not processed for user `a963ee0651124a5bad2c308cac2d210e`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:30.174886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:30.217421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48823f48a90240018ca90fb3643c95ec', 'request_id': 'req-44357622-06c6-420a-82e8-9080522a72f4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.227589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48823f48a90240018ca90fb3643c95ec', 'request_id': 'req-44357622-06c6-420a-82e8-9080522a72f4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.233830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48823f48a90240018ca90fb3643c95ec', 'request_id': 'req-44357622-06c6-420a-82e8-9080522a72f4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.258087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1118/2241] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:30.269028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:30.270638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:30.319516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:30.330392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455e4aa-7bab-42bf-9723-4b3f491872ca None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:30.371032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] The token cache is being invalidate because project 48823f48a90240018ca90fb3643c95ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:30.372353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.373792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.387061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1124/2242] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:29 2022] DELETE /identity/v3/projects/48823f48a90240018ca90fb3643c95ec => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:30.387061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:30.387061 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:30.397907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:30.398582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:30.398885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:30.399187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:30.401145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:30.466571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:30.486723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44357622-06c6-420a-82e8-9080522a72f4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:30.491160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:30.498165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:30.498885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:30.498885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:30.498885 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] PATH_INFO: `/v3/users/8fbae861e3704e5e91a6625b514e45b8` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:30.499148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:30.499239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fbae861e3704e5e91a6625b514e45b8)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:30.499239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:30.503800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1119/2243] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:30.696110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:30.697509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:30.708465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d70e4-e560-450d-b6ed-d7918b03ac54 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:30.712011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:30.712653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:30.713035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:30.713312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:30.715059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:30.778722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:30.789328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:30.789857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1120/2244] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:30.818842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fbae861e3704e5e91a6625b514e45b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.831596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fbae861e3704e5e91a6625b514e45b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.931407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fbae861e3704e5e91a6625b514e45b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.932187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fbae861e3704e5e91a6625b514e45b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.933023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9a8f0c5a-3b93-418a-b466-dd82b7a2f5a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fbae861e3704e5e91a6625b514e45b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:30.950013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1125/2245] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:30 2022] DELETE /identity/v3/users/8fbae861e3704e5e91a6625b514e45b8 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:31.383070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:31.395130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ede8c2-43aa-402a-a42c-d7be84f87ad8 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:31.431809 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:31.459670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:31.460632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:31.460976 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:31.461251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] PATH_INFO: `/v3/projects/257136c5554842f0a9cdaa5b8f1a988b` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:31.462512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=257136c5554842f0a9cdaa5b8f1a988b)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:31.464500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:31.647060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257136c5554842f0a9cdaa5b8f1a988b', 'request_id': 'req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:31.647060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257136c5554842f0a9cdaa5b8f1a988b', 'request_id': 'req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:31.652576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257136c5554842f0a9cdaa5b8f1a988b', 'request_id': 'req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:31.740506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] The token cache is being invalidate because project 257136c5554842f0a9cdaa5b8f1a988b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:31.740798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:31.741420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-107f3b64-d6bc-4e69-9018-ff5dd77e5d38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:31.742966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1121/2246] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:31 2022] DELETE /identity/v3/projects/257136c5554842f0a9cdaa5b8f1a988b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:31.748117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0e736-7c99-4b05-b3ef-4c29b9be86f1 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:31.748608 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0e736-7c99-4b05-b3ef-4c29b9be86f1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:31.748703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0e736-7c99-4b05-b3ef-4c29b9be86f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:31.768483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d9a0e736-7c99-4b05-b3ef-4c29b9be86f1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:31.774455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d9a0e736-7c99-4b05-b3ef-4c29b9be86f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:31.833665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1126/2247] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:31.841999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08ab5-b238-48a6-a84c-7c27cccfcd40 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:31.842414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08ab5-b238-48a6-a84c-7c27cccfcd40 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:31.842607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd08ab5-b238-48a6-a84c-7c27cccfcd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:31.860919 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-8fd08ab5-b238-48a6-a84c-7c27cccfcd40 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:31.862103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8fd08ab5-b238-48a6-a84c-7c27cccfcd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:31.874889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1122/2248] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:31.890832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0e736-7c99-4b05-b3ef-4c29b9be86f1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:31.901392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0e736-7c99-4b05-b3ef-4c29b9be86f1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:31.906201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:31.911102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:31.911102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:31.911102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:31.911102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:31.911102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:31.911102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:31.922351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1127/2249] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:29:31.926725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd08ab5-b238-48a6-a84c-7c27cccfcd40 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:31.936552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd08ab5-b238-48a6-a84c-7c27cccfcd40 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:31.941393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:31.942048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:31.942328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:31.946630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:31.946630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:31.946630 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:32.093307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1123/2250] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:31 2022] POST /identity/v3/projects => generated 363 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:32.096764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:32.107091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b305b-1fca-41b6-b5d8-18d9bfb7f204 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:32.161554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:32.167621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:32.168350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:32.168631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:32.168906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:32.169637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:32.171557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:32.262299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1128/2251] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:32 2022] POST /identity/v3/users => generated 282 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:32.267274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:32.279251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d582e8-9845-488a-a9cb-04f651dc4fd2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:32.284147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:32.284955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:32.285240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:32.285512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:32.286229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:32.287888 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:32.299157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1124/2252] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:32.304191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:32.314242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb6448-0dc9-400b-9ebc-da567d7457ee admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:32.319499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0188b9-7633-4441-b6c6-a659d886414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:32.320181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0188b9-7633-4441-b6c6-a659d886414c admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:32.320467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0188b9-7633-4441-b6c6-a659d886414c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:32.320848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0188b9-7633-4441-b6c6-a659d886414c admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:32.321575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0188b9-7633-4441-b6c6-a659d886414c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:32.323239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0188b9-7633-4441-b6c6-a659d886414c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:32.336201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1129/2253] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:32.341375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:32.352490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69360b9-cca4-408b-b640-51ef30b96601 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:32.358018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:32.362761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:32.362761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:32.362761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] PATH_INFO: `/v3/projects/e8d00b49a01c4abebca9be6489a152c3/users/33973531257e483bbf0504ed7401d1b2/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:32.379366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8d00b49a01c4abebca9be6489a152c3, user_id=33973531257e483bbf0504ed7401d1b2, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:32.382715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:32.484593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1125/2254] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:32 2022] PUT /identity/v3/projects/e8d00b49a01c4abebca9be6489a152c3/users/33973531257e483bbf0504ed7401d1b2/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:32.496960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0686136-009b-4bfe-b087-5cc636eaea1a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:32.497249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0686136-009b-4bfe-b087-5cc636eaea1a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:32.497531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0686136-009b-4bfe-b087-5cc636eaea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:32.525333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d0686136-009b-4bfe-b087-5cc636eaea1a None None] MFA Rules not processed for user `33973531257e483bbf0504ed7401d1b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:32.526538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d0686136-009b-4bfe-b087-5cc636eaea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:32.597736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1130/2255] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:32.609549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:32.611247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:32.674187 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c064d-7308-4cb8-a39c-6eb4999aa6a2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:32.679007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:32.679697 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:32.680015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:32.680291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:32.689545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:32.691005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:32.706372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1126/2256] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:33.430753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0686136-009b-4bfe-b087-5cc636eaea1a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:33.461390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0686136-009b-4bfe-b087-5cc636eaea1a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:33.467477 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6161360-a8ec-4071-97cd-faaf16e6b771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:33.469085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6161360-a8ec-4071-97cd-faaf16e6b771 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:33.469579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6161360-a8ec-4071-97cd-faaf16e6b771 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:33.470054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6161360-a8ec-4071-97cd-faaf16e6b771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:33.472773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f6161360-a8ec-4071-97cd-faaf16e6b771 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:33.539652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6161360-a8ec-4071-97cd-faaf16e6b771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:33.541135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6161360-a8ec-4071-97cd-faaf16e6b771 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:33.550265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1131/2257] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:39.255655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:39.257155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:39.262569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:39.326600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0aa770-e7b6-4d77-a2c7-35ffac8b70b7 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:39.331113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:39.332247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:39.332531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:39.332805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] PATH_INFO: `/v3/users/4b8ec2e7df9046d3a15644b7373302e5` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:39.334028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b8ec2e7df9046d3a15644b7373302e5)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:39.335965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:39.521099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b8ec2e7df9046d3a15644b7373302e5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:39.533997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b8ec2e7df9046d3a15644b7373302e5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:39.632513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b8ec2e7df9046d3a15644b7373302e5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:39.633179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b8ec2e7df9046d3a15644b7373302e5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:39.633793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b8ec2e7df9046d3a15644b7373302e5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:39.647551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1127/2258] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:39 2022] DELETE /identity/v3/users/4b8ec2e7df9046d3a15644b7373302e5 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:39.652075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:39.652574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:39.652715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:39.670505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:39.671830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:39.736431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1132/2259] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:39.746209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:39.747670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:39.805941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61359af5-2b19-40a0-918b-c15864696a39 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:39.809598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:39.810235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:39.810517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:39.810796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:39.819936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:39.821356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:39.831247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1128/2260] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:40.380855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:40.383015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:40.392363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:40.396742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbe601-f615-4e44-b438-85e4e17dcc7c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:40.404055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:40.405015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:40.405299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:40.405599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] PATH_INFO: `/v3/projects/1605a75588584804bcf33e370aee7a6f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:40.407225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1605a75588584804bcf33e370aee7a6f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:40.409599 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:40.523605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1605a75588584804bcf33e370aee7a6f', 'request_id': 'req-97cdb18e-8e00-48a9-946c-7197798223e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:40.530022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1605a75588584804bcf33e370aee7a6f', 'request_id': 'req-97cdb18e-8e00-48a9-946c-7197798223e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:40.536009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1605a75588584804bcf33e370aee7a6f', 'request_id': 'req-97cdb18e-8e00-48a9-946c-7197798223e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:40.707475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] The token cache is being invalidate because project 1605a75588584804bcf33e370aee7a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:40.708313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:40.708933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-97cdb18e-8e00-48a9-946c-7197798223e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:40.710593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1133/2261] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:40 2022] DELETE /identity/v3/projects/1605a75588584804bcf33e370aee7a6f => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:40.714541 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:40.724511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafcdb75-4ee9-4fa3-8366-4250084515f4 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:40.745861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:40.786971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:40.788900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:40.789324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:40.793616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] PATH_INFO: `/v3/users/a963ee0651124a5bad2c308cac2d210e` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:40.793616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a963ee0651124a5bad2c308cac2d210e)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:40.793616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:41.008133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a963ee0651124a5bad2c308cac2d210e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.022491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a963ee0651124a5bad2c308cac2d210e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.054052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a89beb-3e23-4cd8-82bd-5d33e3e9e6a7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:41.054949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a89beb-3e23-4cd8-82bd-5d33e3e9e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:41.055225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a89beb-3e23-4cd8-82bd-5d33e3e9e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:41.077060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-91a89beb-3e23-4cd8-82bd-5d33e3e9e6a7 None None] MFA Rules not processed for user `ee4307a7bdf84fe18efd6ba3dbe50c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:41.078309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-91a89beb-3e23-4cd8-82bd-5d33e3e9e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:41.129522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a963ee0651124a5bad2c308cac2d210e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.130180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a963ee0651124a5bad2c308cac2d210e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.130847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a963ee0651124a5bad2c308cac2d210e'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.146190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1129/2262] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:40 2022] DELETE /identity/v3/users/a963ee0651124a5bad2c308cac2d210e => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:41.151015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:41.152586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:41.152837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1134/2263] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:41.226773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22aedb3-26a0-4033-adc9-6dd9f5bb4181 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:41.232188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:41.233121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:41.233628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:41.233843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] PATH_INFO: `/v3/users/1727f0c55a1d46c2b693e3ea8bfada9a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:41.234892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1727f0c55a1d46c2b693e3ea8bfada9a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:41.236942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:41.396051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1727f0c55a1d46c2b693e3ea8bfada9a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.410424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1727f0c55a1d46c2b693e3ea8bfada9a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.494406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:41.518270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1727f0c55a1d46c2b693e3ea8bfada9a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.518947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1727f0c55a1d46c2b693e3ea8bfada9a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.519811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1727f0c55a1d46c2b693e3ea8bfada9a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.533817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1130/2264] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:41 2022] DELETE /identity/v3/users/1727f0c55a1d46c2b693e3ea8bfada9a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:41.538873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:41.539276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:41.539566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:41.559017 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:41.560270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:41.622450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:41.625074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:41.632266 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1135/2265] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:41.644845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:41.647068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:41.656113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592f4b53-a369-4702-8308-bc2a9ec13728 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:41.661273 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:41.662749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:41.662749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:41.662954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] PATH_INFO: `/v3/projects/376e09b38dcb4e619aa92dd021a10e13` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:41.664166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] RBAC: Authorizing `identity:delete_project(project_id=376e09b38dcb4e619aa92dd021a10e13)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:41.666173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:41.701959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:41.727456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7387b06-186e-4949-b35b-8d7a74774e14 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:41.731263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ddbd87-6bb6-42aa-a095-937ed71da3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:41.731951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddbd87-6bb6-42aa-a095-937ed71da3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:41.732235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddbd87-6bb6-42aa-a095-937ed71da3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:41.732520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ddbd87-6bb6-42aa-a095-937ed71da3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:41.742188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ddbd87-6bb6-42aa-a095-937ed71da3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:41.743625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ddbd87-6bb6-42aa-a095-937ed71da3c0 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:41.753513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1136/2266] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:41.786610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376e09b38dcb4e619aa92dd021a10e13', 'request_id': 'req-f754020a-f3a4-45c6-91c5-4e5945f93035'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.793602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376e09b38dcb4e619aa92dd021a10e13', 'request_id': 'req-f754020a-f3a4-45c6-91c5-4e5945f93035'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.799574 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376e09b38dcb4e619aa92dd021a10e13', 'request_id': 'req-f754020a-f3a4-45c6-91c5-4e5945f93035'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.870957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] The token cache is being invalidate because project 376e09b38dcb4e619aa92dd021a10e13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:41.871256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.872987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f754020a-f3a4-45c6-91c5-4e5945f93035 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:41.873548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1131/2267] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:41 2022] DELETE /identity/v3/projects/376e09b38dcb4e619aa92dd021a10e13 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:41.879543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4558ff-b928-4ece-8d30-199f111cf63e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:41.880376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4558ff-b928-4ece-8d30-199f111cf63e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:41.880735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4558ff-b928-4ece-8d30-199f111cf63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:41.901681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6a4558ff-b928-4ece-8d30-199f111cf63e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:41.902673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6a4558ff-b928-4ece-8d30-199f111cf63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:41.989738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1137/2268] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:41.997443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99329e3-b136-4f29-aad6-576153c0dd2d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:41.998329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99329e3-b136-4f29-aad6-576153c0dd2d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:41.998694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99329e3-b136-4f29-aad6-576153c0dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.032963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f99329e3-b136-4f29-aad6-576153c0dd2d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:42.035690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f99329e3-b136-4f29-aad6-576153c0dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:42.043307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1132/2269] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:42.047459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4558ff-b928-4ece-8d30-199f111cf63e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.058547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4558ff-b928-4ece-8d30-199f111cf63e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.064273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.065713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.065935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.066220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.068390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:42.068390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.075870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.083710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1138/2270] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:29:42.088461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99329e3-b136-4f29-aad6-576153c0dd2d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.098978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99329e3-b136-4f29-aad6-576153c0dd2d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.106289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.106913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.107384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.107665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.108399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.110597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.277041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1133/2271] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:42 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:42.279635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.289864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb668e-d673-43c7-9bd9-d5cca92bf431 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.312307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.314225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:42.375139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.378203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.378203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.378203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.378203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.383333 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.402587 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c4d17-5dae-408e-b653-0d853e937d22 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.409450 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.409753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.409753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.410033 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] PATH_INFO: `/v3/projects/dc5e828b9b8f44c7b578480966ea4cd2` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.411255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5e828b9b8f44c7b578480966ea4cd2)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.413203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.443722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:42.559760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1139/2272] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:42 2022] POST /identity/v3/users => generated 294 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:42.568626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.586705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3357340d-b5f9-4f8d-af15-0f72eab059da admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.591432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.592141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.592428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.592706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.593495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.595084 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.605935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1140/2273] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:42.610150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.620463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5e828b9b8f44c7b578480966ea4cd2', 'request_id': 'req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:42.620683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5440e-5706-4e8c-a35d-d17b808bdb62 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.626850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5e828b9b8f44c7b578480966ea4cd2', 'request_id': 'req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:42.633394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5e828b9b8f44c7b578480966ea4cd2', 'request_id': 'req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:42.661398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:42.679809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.680483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.680769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.681064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.681792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.683402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.693395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1141/2274] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:42.698678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.708855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a8773-087c-4e66-8fae-013a0c511d33 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.713648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0629478-274b-441c-a39a-b2ffea508c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.714534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0629478-274b-441c-a39a-b2ffea508c9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.714836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0629478-274b-441c-a39a-b2ffea508c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.715215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0629478-274b-441c-a39a-b2ffea508c9e admin admin] PATH_INFO: `/v3/projects/7a43a07a087a409fad017e3eeb950810/users/a40513946fdf46c6a0be15651f46bf68/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.730308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0629478-274b-441c-a39a-b2ffea508c9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a43a07a087a409fad017e3eeb950810, user_id=a40513946fdf46c6a0be15651f46bf68, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.733015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0629478-274b-441c-a39a-b2ffea508c9e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.764062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] The token cache is being invalidate because project dc5e828b9b8f44c7b578480966ea4cd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:42.764353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:42.764987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:42.766566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1134/2275] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:42 2022] DELETE /identity/v3/projects/dc5e828b9b8f44c7b578480966ea4cd2 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:42.770363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.771850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:42.784793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417bd2b-1c0b-4e8a-bf5b-a328fa41280d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:42.789812 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:42.790727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.791015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.791295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] PATH_INFO: `/v3/users/e9f7bf9e901f4ab8a8032f5d597caa2a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.792532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f7bf9e901f4ab8a8032f5d597caa2a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:42.802941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:42.861465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1142/2276] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:42 2022] PUT /identity/v3/projects/7a43a07a087a409fad017e3eeb950810/users/a40513946fdf46c6a0be15651f46bf68/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:42.873554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:42.873843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:42.874139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:42.901535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] MFA Rules not processed for user `a40513946fdf46c6a0be15651f46bf68`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:42.902754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:42.966958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1143/2277] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:42.971356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:42.972707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:43.025629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f7bf9e901f4ab8a8032f5d597caa2a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.039890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f7bf9e901f4ab8a8032f5d597caa2a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.044956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bf1c3b-4435-4c88-8bc6-c152d6ce3353 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:43.049869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:43.050549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.050818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.051090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.051840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:43.054671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:43.112053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f7bf9e901f4ab8a8032f5d597caa2a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.112610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f7bf9e901f4ab8a8032f5d597caa2a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.113211 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f7bf9e901f4ab8a8032f5d597caa2a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.126573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1135/2278] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:42 2022] DELETE /identity/v3/users/e9f7bf9e901f4ab8a8032f5d597caa2a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:43.148457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1144/2279] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:42 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:43.152606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:43.154022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:43.235981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c862a5-c20c-4b47-8ba7-b1d416d4b993 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:43.293249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:43.293992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.294329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.294607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.295294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:43.297289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:43.387821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1136/2280] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:43 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:43.391504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:43.404304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d4b41-7279-49b9-a8c0-829154f0287f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:43.409464 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:43.410199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.413418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.413418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.413418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:43.413773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:43.425004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1145/2281] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:43.429474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:43.456107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b0bfcc-f21a-4e71-82fc-1d32ce0c84fb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:43.468823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:43.469898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.471440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.471754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] PATH_INFO: `/v3/projects/173df54374284f9cba172b54624443fb/users/8b2bf962843c4457984ecdc6c1ee74a0/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.494335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173df54374284f9cba172b54624443fb, user_id=8b2bf962843c4457984ecdc6c1ee74a0, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:43.497388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:43.669100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:43.670905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:43.684876 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0a7d8-bf07-4ab1-873d-a9c1cd085907 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:43.689735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:43.690613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.690899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.691180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] PATH_INFO: `/v3/projects/c91c9e9b15ab4a0889e15494ae781ca7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.697584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91c9e9b15ab4a0889e15494ae781ca7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:43.697584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:43.754718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1137/2282] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:43 2022] PUT /identity/v3/projects/173df54374284f9cba172b54624443fb/users/8b2bf962843c4457984ecdc6c1ee74a0/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:43.757521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:43.821868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:43.824154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df72121-b913-4b33-a3aa-63c54629f819 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:43.828707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:43.829375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.829662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.829939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.830692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:43.832335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:43.842578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1138/2283] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:43.846641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:43.856705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a2a3c4-7cb0-43e3-ad40-4bf818b72ff5 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:43.861334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd3fbd-8285-4c3b-880b-0fb234308053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:43.862244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd3fbd-8285-4c3b-880b-0fb234308053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.862526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd3fbd-8285-4c3b-880b-0fb234308053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.862799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd3fbd-8285-4c3b-880b-0fb234308053 admin admin] PATH_INFO: `/v3/projects/173df54374284f9cba172b54624443fb/users/8b2bf962843c4457984ecdc6c1ee74a0/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.864772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd3fbd-8285-4c3b-880b-0fb234308053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=173df54374284f9cba172b54624443fb, user_id=8b2bf962843c4457984ecdc6c1ee74a0, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:43.867486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd3fbd-8285-4c3b-880b-0fb234308053 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:43.877681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1139/2284] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:43 2022] PUT /identity/v3/projects/173df54374284f9cba172b54624443fb/users/8b2bf962843c4457984ecdc6c1ee74a0/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:43.887895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:43.888185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:43.890799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:43.896284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91c9e9b15ab4a0889e15494ae781ca7', 'request_id': 'req-c6678ec7-687a-4877-9382-46c1203a1cca'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.903037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91c9e9b15ab4a0889e15494ae781ca7', 'request_id': 'req-c6678ec7-687a-4877-9382-46c1203a1cca'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.908658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91c9e9b15ab4a0889e15494ae781ca7', 'request_id': 'req-c6678ec7-687a-4877-9382-46c1203a1cca'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:43.916050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:43.917740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] MFA Rules not processed for user `8b2bf962843c4457984ecdc6c1ee74a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:43.918969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:43.987363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1140/2285] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:43.997982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:43.999387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:44.037506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] The token cache is being invalidate because project c91c9e9b15ab4a0889e15494ae781ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:44.038065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.038708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.040458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1146/2286] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:43 2022] DELETE /identity/v3/projects/c91c9e9b15ab4a0889e15494ae781ca7 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:44.044465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:44.046355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:44.057053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccf0b5-5657-4e7a-bbdb-6935e1e54c73 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:44.061027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:44.061687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:44.061966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:44.062239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:44.064039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:44.076605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:44.078014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:44.088332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1141/2287] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:44.123328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6678ec7-687a-4877-9382-46c1203a1cca admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:44.144865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:44.145973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:44.146547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:44.146982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] PATH_INFO: `/v3/users/2c7d1d0513334de398d6c8fccd87dfb2` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:44.149228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7d1d0513334de398d6c8fccd87dfb2)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:44.152030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:44.246059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:44.247518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:44.262354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce4f39-5394-47e2-b2f6-bb55de702026 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:44.267386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:44.268314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:44.268601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:44.268876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] PATH_INFO: `/v3/users/a40513946fdf46c6a0be15651f46bf68` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:44.270573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a40513946fdf46c6a0be15651f46bf68)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:44.275554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:44.417625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7d1d0513334de398d6c8fccd87dfb2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.432050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7d1d0513334de398d6c8fccd87dfb2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.487919 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a40513946fdf46c6a0be15651f46bf68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.500455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7d1d0513334de398d6c8fccd87dfb2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.501136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7d1d0513334de398d6c8fccd87dfb2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.501752 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-079fe1a3-5ec2-43e9-95df-a20170fa3789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7d1d0513334de398d6c8fccd87dfb2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.502568 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a40513946fdf46c6a0be15651f46bf68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.516267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1147/2288] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:44 2022] DELETE /identity/v3/users/2c7d1d0513334de398d6c8fccd87dfb2 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:44.598402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:44.599578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a40513946fdf46c6a0be15651f46bf68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.600633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a40513946fdf46c6a0be15651f46bf68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.602057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a40513946fdf46c6a0be15651f46bf68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:44.615777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1142/2289] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:44 2022] DELETE /identity/v3/users/a40513946fdf46c6a0be15651f46bf68 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:44.620941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:44.621227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:44.621503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:44.638527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:44.639886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:44.653226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:44.707616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1148/2290] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:44.712373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:44.713778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:44.774571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cfcfad-26dd-4cdb-9e4c-a1a3f4daa2d8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:44.778205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:44.778859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:44.779193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:44.779475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:44.790039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:44.791821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:44.802516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1143/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:45.136787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:45.139338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:45.155619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b980b1f-bb4d-46ef-bf9a-f7c14950856e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:45.161284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:45.162056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.162343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.162611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] PATH_INFO: `/v3/projects/5fb0020ce57743cab1ec0e4efc27d7e9` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.164453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fb0020ce57743cab1ec0e4efc27d7e9)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:45.166773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:45.268807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:45.270300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:45.293070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568b690-b23a-4392-85e9-dfb49b641961 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:45.329062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:45.330131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fb0020ce57743cab1ec0e4efc27d7e9', 'request_id': 'req-364bed5f-928e-4e10-bf8b-57777f29acb4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.370678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fb0020ce57743cab1ec0e4efc27d7e9', 'request_id': 'req-364bed5f-928e-4e10-bf8b-57777f29acb4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.414091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fb0020ce57743cab1ec0e4efc27d7e9', 'request_id': 'req-364bed5f-928e-4e10-bf8b-57777f29acb4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.416147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:45.418067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.418268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.418558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] PATH_INFO: `/v3/projects/7a43a07a087a409fad017e3eeb950810` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.420462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a43a07a087a409fad017e3eeb950810)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:45.422389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:45.508594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] The token cache is being invalidate because project 5fb0020ce57743cab1ec0e4efc27d7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:45.515192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.515192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-364bed5f-928e-4e10-bf8b-57777f29acb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.515192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1149/2292] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:45 2022] DELETE /identity/v3/projects/5fb0020ce57743cab1ec0e4efc27d7e9 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:45.520697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1c3f-c388-4305-84a6-7bb306de9abf None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.520985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1c3f-c388-4305-84a6-7bb306de9abf None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.521265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1c3f-c388-4305-84a6-7bb306de9abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.541498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-eafc1c3f-c388-4305-84a6-7bb306de9abf None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:45.543109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-eafc1c3f-c388-4305-84a6-7bb306de9abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:45.552576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1150/2293] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:45.561183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5bd5-a2d0-4046-9f4b-b2bc6a345cd7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.561471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5bd5-a2d0-4046-9f4b-b2bc6a345cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.561751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ab5bd5-a2d0-4046-9f4b-b2bc6a345cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.580797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-06ab5bd5-a2d0-4046-9f4b-b2bc6a345cd7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:45.582032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-06ab5bd5-a2d0-4046-9f4b-b2bc6a345cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:45.590950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a43a07a087a409fad017e3eeb950810', 'request_id': 'req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.604981 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a43a07a087a409fad017e3eeb950810', 'request_id': 'req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.612908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a43a07a087a409fad017e3eeb950810', 'request_id': 'req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.657251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1151/2294] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:45.661457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab5bd5-a2d0-4046-9f4b-b2bc6a345cd7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:45.674165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ab5bd5-a2d0-4046-9f4b-b2bc6a345cd7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:45.679271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:45.679974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.680255 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.680527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.681252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:45.681554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:45.683243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:45.696330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1152/2295] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:29:45.700584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:45.713413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d888147-c6c5-4be9-9ba7-d1ca1a474145 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:45.718292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:45.719011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.719430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.719735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.722620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:45.724597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:45.730234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] The token cache is being invalidate because project 7a43a07a087a409fad017e3eeb950810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:45.730608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.731511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:45.733440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1144/2296] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:45 2022] DELETE /identity/v3/projects/7a43a07a087a409fad017e3eeb950810 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:45.737197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:45.738625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:45.751268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c5fc9-aa38-4bd1-b8f3-4bd6a48796d2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:45.755991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:45.756870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.757183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.757500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] PATH_INFO: `/v3/users/8b2bf962843c4457984ecdc6c1ee74a0` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.759614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2bf962843c4457984ecdc6c1ee74a0)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:45.760721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:45.862918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1153/2297] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:45 2022] POST /identity/v3/projects => generated 383 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:45.866769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:45.868773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:45.873769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:45.934621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc035f7d-91f0-4f22-9d89-2308512618df admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:45.939292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:45.939992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:45.940272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:45.940545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:45.941233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:45.943148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:46.034901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2bf962843c4457984ecdc6c1ee74a0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:46.049041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2bf962843c4457984ecdc6c1ee74a0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:46.105524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1154/2298] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:45 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:46.106819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:46.117414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc51fc3-24a8-41af-bb56-0b8ffca8c8fe admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:46.123279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:46.123991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:46.124444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:46.124728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:46.125470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:46.127523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:46.143203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1155/2299] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:46.144332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:46.157499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad458d-ee30-4520-967c-44fb4ff889af admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:46.158144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2bf962843c4457984ecdc6c1ee74a0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:46.158775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2bf962843c4457984ecdc6c1ee74a0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:46.159550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2bf962843c4457984ecdc6c1ee74a0'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:46.163621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393bfedb-1cf4-413e-b7c2-8ebbb923a7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:46.164388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bfedb-1cf4-413e-b7c2-8ebbb923a7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:46.164684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bfedb-1cf4-413e-b7c2-8ebbb923a7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:46.167590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393bfedb-1cf4-413e-b7c2-8ebbb923a7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:46.167590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bfedb-1cf4-413e-b7c2-8ebbb923a7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:46.168076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393bfedb-1cf4-413e-b7c2-8ebbb923a7e4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:46.175265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1145/2300] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:45 2022] DELETE /identity/v3/users/8b2bf962843c4457984ecdc6c1ee74a0 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:46.181322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1156/2301] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:46.185719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:46.187406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:46.256304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62391a80-dfe8-49d7-b3c6-dd233f871700 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:46.262755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:46.263579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:46.263902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:46.264182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] PATH_INFO: `/v3/projects/1e8158a7cacc4b60876f0bc7ab9a20fb/users/da608b807dc64a63b2703335e36f9e4a/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:46.282282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8158a7cacc4b60876f0bc7ab9a20fb, user_id=da608b807dc64a63b2703335e36f9e4a, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:46.284088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:46.410229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1146/2302] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:46 2022] PUT /identity/v3/projects/1e8158a7cacc4b60876f0bc7ab9a20fb/users/da608b807dc64a63b2703335e36f9e4a/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:46.412721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:46.413190 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:46.413713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:46.496297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] MFA Rules not processed for user `da608b807dc64a63b2703335e36f9e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:46.511534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:46.529593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:46.644673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1157/2303] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:46.648896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:46.749453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ce6a67-d044-4591-8bbf-4a2ee286ab24 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:46.756824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:46.757917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:46.758932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:46.759386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:46.760315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:46.762464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:46.920691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:46.922839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:46.924475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1147/2304] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:46 2022] POST /identity/v3/projects => generated 381 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:46.929717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:46.950643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacb2e8-4a80-4efa-bd33-e36974eeec51 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.007605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cfcc9-7f1f-4cd1-9ce6-97ad3dd91a63 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.013612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.014685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.015126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.015572 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.016322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.018737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.021559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.022487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.022902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.023278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] PATH_INFO: `/v3/projects/173df54374284f9cba172b54624443fb` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.024647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=173df54374284f9cba172b54624443fb)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.026691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.207613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1148/2305] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:46 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:47.212208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.222870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11369b63-5466-4fbf-a55f-1913626ea47c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.223972 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173df54374284f9cba172b54624443fb', 'request_id': 'req-7ad45191-a180-4f41-b3d3-91cf111dc0e2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:47.231243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173df54374284f9cba172b54624443fb', 'request_id': 'req-7ad45191-a180-4f41-b3d3-91cf111dc0e2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:47.238691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173df54374284f9cba172b54624443fb', 'request_id': 'req-7ad45191-a180-4f41-b3d3-91cf111dc0e2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:47.312112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.312792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.313076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.313354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.314081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.315791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.325965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1149/2306] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:47.328856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] The token cache is being invalidate because project 173df54374284f9cba172b54624443fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:47.329222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:47.330026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7ad45191-a180-4f41-b3d3-91cf111dc0e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:47.330141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.331545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1158/2307] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:46 2022] DELETE /identity/v3/projects/173df54374284f9cba172b54624443fb => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:47.331762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:47.336741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c985e-404f-4f98-8cdb-3030aba6278b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.337053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c985e-404f-4f98-8cdb-3030aba6278b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.337381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c985e-404f-4f98-8cdb-3030aba6278b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.345102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba246406-547f-4da5-a745-ab41c22ff9ac admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.349756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.351228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.351526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.351915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] PATH_INFO: `/v3/projects/cd9af4b7d95e492cad7f4b6a9bebc7c5/users/cbee6b88566d4db281b43731eba20946/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.356671 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-381c985e-404f-4f98-8cdb-3030aba6278b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:47.357928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-381c985e-404f-4f98-8cdb-3030aba6278b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:47.366716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1159/2308] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:47.367269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9af4b7d95e492cad7f4b6a9bebc7c5, user_id=cbee6b88566d4db281b43731eba20946, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.369980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.374835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25d177-ba89-40cb-9157-b348e796ce1d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.375126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25d177-ba89-40cb-9157-b348e796ce1d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.375411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25d177-ba89-40cb-9157-b348e796ce1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.392950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ac25d177-ba89-40cb-9157-b348e796ce1d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:47.394145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ac25d177-ba89-40cb-9157-b348e796ce1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:47.402617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1160/2309] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:47.406545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25d177-ba89-40cb-9157-b348e796ce1d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.416303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25d177-ba89-40cb-9157-b348e796ce1d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.422156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.423982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.423982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.423982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.424304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:47.424466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.426090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.439137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1161/2310] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:29:47.444980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.450369 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1150/2311] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:47 2022] PUT /identity/v3/projects/cd9af4b7d95e492cad7f4b6a9bebc7c5/users/cbee6b88566d4db281b43731eba20946/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:47.454319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.456262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84063255-3ff5-4804-9dcb-44f2a0926326 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.520878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.521561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.521849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.522130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.523534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.525541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.527329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec1b1a3-1462-46db-8c07-a2090c656dd6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.532905 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.533862 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.534406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.534734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.535633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.537511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.549673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1151/2312] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:47.554128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.564414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c864d-7de2-413c-9096-743fd6de3ece admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.570753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d005ed1-6928-4b48-a66f-c72fcbd277d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.571805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d005ed1-6928-4b48-a66f-c72fcbd277d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.572174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d005ed1-6928-4b48-a66f-c72fcbd277d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.572523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d005ed1-6928-4b48-a66f-c72fcbd277d5 admin admin] PATH_INFO: `/v3/projects/cd9af4b7d95e492cad7f4b6a9bebc7c5/users/cbee6b88566d4db281b43731eba20946/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.574642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d005ed1-6928-4b48-a66f-c72fcbd277d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9af4b7d95e492cad7f4b6a9bebc7c5, user_id=cbee6b88566d4db281b43731eba20946, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.577456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d005ed1-6928-4b48-a66f-c72fcbd277d5 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.594120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1152/2313] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:47 2022] PUT /identity/v3/projects/cd9af4b7d95e492cad7f4b6a9bebc7c5/users/cbee6b88566d4db281b43731eba20946/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:47.607013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.607489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.607584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.637744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] MFA Rules not processed for user `cbee6b88566d4db281b43731eba20946`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:47.639034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:47.653917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1162/2314] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:47 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:47.658052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.718198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1153/2315] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:47 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:47.724483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:47.730022 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.731581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:47.741570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4bcbe0-47aa-436b-a6e2-541255821496 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.745965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.746638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.746921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.747236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.747969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.749940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.790953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1762cfc-f184-4514-9af6-385b771ab437 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.841998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf16c5-e2f5-4582-b60d-d5269530b548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.842978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf16c5-e2f5-4582-b60d-d5269530b548 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.843427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf16c5-e2f5-4582-b60d-d5269530b548 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.843939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf16c5-e2f5-4582-b60d-d5269530b548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.857222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1163/2316] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:47 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:47.858384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.870009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423725bb-ab10-4ba1-88ca-b4bd15c7f3c1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.875114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.876282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.876282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.876557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.877290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.878900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.892898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1164/2317] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:47.895582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.906297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a6a49-a501-46bb-a582-2137c9a133c5 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.912550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.913352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.914050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.914436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.915307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.917920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.919410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf16c5-e2f5-4582-b60d-d5269530b548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.920899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf16c5-e2f5-4582-b60d-d5269530b548 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:47.933970 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1165/2318] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:47.934139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1154/2319] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:47 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:47.935325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:47.947038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca54dc1-f2b8-4ec6-b365-e20782755a39 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:47.952541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:47.953941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:47.954272 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:47.954622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] PATH_INFO: `/v3/projects/a9fa896620464a6cae07f2117441c93f/users/2aa57242371b4fcfb99be2710706c16b/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:47.970506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fa896620464a6cae07f2117441c93f, user_id=2aa57242371b4fcfb99be2710706c16b, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:47.973344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:48.066092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1166/2320] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:47 2022] PUT /identity/v3/projects/a9fa896620464a6cae07f2117441c93f/users/2aa57242371b4fcfb99be2710706c16b/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:48.077883 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:48.078181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:48.078462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:48.095986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:48.108757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] MFA Rules not processed for user `2aa57242371b4fcfb99be2710706c16b`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:48.109986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:48.153495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd73f50c-a03b-42ac-8028-6577749286e1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:48.157415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ba6eb-ebba-485f-b092-3a9e80f53ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:48.158174 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba6eb-ebba-485f-b092-3a9e80f53ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:48.158461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba6eb-ebba-485f-b092-3a9e80f53ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:48.158738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ba6eb-ebba-485f-b092-3a9e80f53ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:48.160575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cb0ba6eb-ebba-485f-b092-3a9e80f53ee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:48.176098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1155/2321] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:48.189122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:48.190531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:48.205004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00b5366-3652-481a-b8bb-671c65ed5226 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:48.208890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:48.212430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:48.212430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:48.212430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:48.218986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:48.220506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:48.223790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ba6eb-ebba-485f-b092-3a9e80f53ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:48.225279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ba6eb-ebba-485f-b092-3a9e80f53ee0 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:48.231365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1156/2322] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:48.233388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:48.235835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1167/2323] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:48.258254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:48.334129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ace88-1c7a-453e-8883-bac9670c910c service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:48.342470 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:48.343864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:48.344156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:48.344474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] PATH_INFO: `/v3/users/2aa57242371b4fcfb99be2710706c16b` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:48.345785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aa57242371b4fcfb99be2710706c16b)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:48.347914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:48.478654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:48.530937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aa57242371b4fcfb99be2710706c16b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:48.544731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aa57242371b4fcfb99be2710706c16b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:48.661766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aa57242371b4fcfb99be2710706c16b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:48.662405 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aa57242371b4fcfb99be2710706c16b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:48.663857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aa57242371b4fcfb99be2710706c16b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:48.677445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1157/2324] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:48 2022] DELETE /identity/v3/users/2aa57242371b4fcfb99be2710706c16b => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:48.682015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:48.682311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:48.682593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:48.707650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:48.707650 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:48.762955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1168/2325] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:48.773662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:48.776257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:48.817336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:48.818782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:48.830486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bc300-f331-4f45-a554-4f05418792c2 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:48.835759 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:48.836702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:48.836937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:48.837208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] PATH_INFO: `/v3/users/da608b807dc64a63b2703335e36f9e4a` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:48.837946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c474b-6d4f-4aba-9396-482f1537fac2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:48.838403 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da608b807dc64a63b2703335e36f9e4a)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:48.840373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:48.841643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa95a5-9eac-481e-9638-204d26a17c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:48.842332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa95a5-9eac-481e-9638-204d26a17c3f service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:48.842598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa95a5-9eac-481e-9638-204d26a17c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:48.842871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa95a5-9eac-481e-9638-204d26a17c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:48.864729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa95a5-9eac-481e-9638-204d26a17c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:48.867151 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:48.868934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa95a5-9eac-481e-9638-204d26a17c3f service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:48.879449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1158/2326] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:49.053604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da608b807dc64a63b2703335e36f9e4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.066721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da608b807dc64a63b2703335e36f9e4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.110343 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:49.135179 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da608b807dc64a63b2703335e36f9e4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.135835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da608b807dc64a63b2703335e36f9e4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.136456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da608b807dc64a63b2703335e36f9e4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.147358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:49.152354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1169/2327] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:48 2022] DELETE /identity/v3/users/da608b807dc64a63b2703335e36f9e4a => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:49.157240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:49.157782 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:49.158060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:49.177382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:49.178604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:49.226179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:49.250333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1159/2328] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:49.261195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:49.262598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:49.290469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:49.291961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:49.307723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac620ab2-ce5a-4717-bf98-e811a71c1d6f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:49.314214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:49.315481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:49.316391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:49.316735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] PATH_INFO: `/v3/projects/a9fa896620464a6cae07f2117441c93f` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:49.318303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9fa896620464a6cae07f2117441c93f)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:49.329575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:49.332383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918deda-87a2-4103-9dd2-d1c391395766 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:49.337470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ed6a6-1b57-4085-8c63-77982af07d44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:49.338249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed6a6-1b57-4085-8c63-77982af07d44 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:49.338851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed6a6-1b57-4085-8c63-77982af07d44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:49.338851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ed6a6-1b57-4085-8c63-77982af07d44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:49.350180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ed6a6-1b57-4085-8c63-77982af07d44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:49.351886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793ed6a6-1b57-4085-8c63-77982af07d44 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:49.368196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1170/2329] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:49.430801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9fa896620464a6cae07f2117441c93f', 'request_id': 'req-736b0dc7-8454-45e3-9e4b-a27787eb0241'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.437459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9fa896620464a6cae07f2117441c93f', 'request_id': 'req-736b0dc7-8454-45e3-9e4b-a27787eb0241'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.443795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9fa896620464a6cae07f2117441c93f', 'request_id': 'req-736b0dc7-8454-45e3-9e4b-a27787eb0241'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.592652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] The token cache is being invalidate because project a9fa896620464a6cae07f2117441c93f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:49.593026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.593689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-736b0dc7-8454-45e3-9e4b-a27787eb0241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:49.595533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1160/2330] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:49 2022] DELETE /identity/v3/projects/a9fa896620464a6cae07f2117441c93f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:29:49.603140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b88de-4acf-4a15-bf89-02b09658fbb0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:49.603652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b88de-4acf-4a15-bf89-02b09658fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:49.604176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b88de-4acf-4a15-bf89-02b09658fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:49.635901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-084b88de-4acf-4a15-bf89-02b09658fbb0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:49.638238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-084b88de-4acf-4a15-bf89-02b09658fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:49.716099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1171/2331] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:49.723954 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a0262-9913-4ad4-9663-c92d15e7101d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:49.724250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a0262-9913-4ad4-9663-c92d15e7101d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:49.724532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a0262-9913-4ad4-9663-c92d15e7101d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:49.744001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-049a0262-9913-4ad4-9663-c92d15e7101d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:49.746303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-049a0262-9913-4ad4-9663-c92d15e7101d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:49.758324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1161/2332] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:49.761348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b88de-4acf-4a15-bf89-02b09658fbb0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:49.768027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:49.773879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b88de-4acf-4a15-bf89-02b09658fbb0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:49.779925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:49.780684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:49.780801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:49.781054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:49.781790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:49.782094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:49.783720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:49.799398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1172/2333] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:29:49.805046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a0262-9913-4ad4-9663-c92d15e7101d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:49.814690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a0262-9913-4ad4-9663-c92d15e7101d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:49.819430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:49.820096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:49.820378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:49.821271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:49.821390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:49.823258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:49.920937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:49.924080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:49.944759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1162/2334] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:49 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:49.947257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac776dd-c331-4abd-bbd4-4440c6a91554 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:49.949291 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:50.014332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:50.015009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:50.015375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.015665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.015988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] PATH_INFO: `/v3/projects/1e8158a7cacc4b60876f0bc7ab9a20fb` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.017482 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8158a7cacc4b60876f0bc7ab9a20fb)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:50.019148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e08fcff-dda9-4c19-8db7-91a378909f21 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:50.019461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:50.023977 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:50.024673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.024957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.025236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.026253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:50.027901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:50.143937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1163/2335] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:49 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:29:50.145018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:50.153259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8158a7cacc4b60876f0bc7ab9a20fb', 'request_id': 'req-6d3d3fd5-4129-4f46-ad44-1150096a412b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.155193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3960bf91-ff20-41ed-a1a1-95d83a625888 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:50.160082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8158a7cacc4b60876f0bc7ab9a20fb', 'request_id': 'req-6d3d3fd5-4129-4f46-ad44-1150096a412b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.169252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8158a7cacc4b60876f0bc7ab9a20fb', 'request_id': 'req-6d3d3fd5-4129-4f46-ad44-1150096a412b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.212431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:50.213110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.213395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.213673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.214401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:50.216008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:50.226962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1164/2336] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:50.230735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:50.239207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] The token cache is being invalidate because project 1e8158a7cacc4b60876f0bc7ab9a20fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:50.239500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.240281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.242012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1173/2337] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:49 2022] DELETE /identity/v3/projects/1e8158a7cacc4b60876f0bc7ab9a20fb => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:50.243655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63739e54-2f92-43c2-982e-846b6e377f49 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:50.246207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:50.247585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:50.250214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:50.251080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.251628 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.251945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.252679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:50.254295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:50.260612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d3fd5-4129-4f46-ad44-1150096a412b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:50.265008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1165/2338] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:50.265635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:50.267098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.267541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.267845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] PATH_INFO: `/v3/users/cbee6b88566d4db281b43731eba20946` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.269591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbee6b88566d4db281b43731eba20946)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:50.270072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:50.271540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:50.272026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:50.286989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2234f56-f684-40cd-8248-f31dbb0d39e5 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:50.294138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00509e-f041-43c6-aecc-607116e43758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:50.295041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00509e-f041-43c6-aecc-607116e43758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.295805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00509e-f041-43c6-aecc-607116e43758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.295805 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00509e-f041-43c6-aecc-607116e43758 admin admin] PATH_INFO: `/v3/projects/771d85ed5c3b4a6bbc9945d88cfcbab8/users/b9537b7add3b4b5da501903db49a2a11/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.313936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00509e-f041-43c6-aecc-607116e43758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=771d85ed5c3b4a6bbc9945d88cfcbab8, user_id=b9537b7add3b4b5da501903db49a2a11, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:50.316673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba00509e-f041-43c6-aecc-607116e43758 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:50.386757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1166/2339] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:50 2022] PUT /identity/v3/projects/771d85ed5c3b4a6bbc9945d88cfcbab8/users/b9537b7add3b4b5da501903db49a2a11/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:50.391815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.392111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.392389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.410079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:50.411380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:50.465462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbee6b88566d4db281b43731eba20946'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.474255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1167/2340] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:50.481890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbee6b88566d4db281b43731eba20946'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.484251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:50.485692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:50.539178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:50.542991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99604fe4-503e-4972-b99b-93ed2a16e83b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:50.546740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:50.547387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:50.547673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:50.547978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:50.558471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:50.564938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:50.570560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1168/2341] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:29:50.581183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbee6b88566d4db281b43731eba20946'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.581824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbee6b88566d4db281b43731eba20946'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.582458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-7cc28bee-db08-46d4-bc59-9f951d1c1716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbee6b88566d4db281b43731eba20946'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:50.584958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:50.597316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1174/2342] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:29:50 2022] DELETE /identity/v3/users/cbee6b88566d4db281b43731eba20946 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:51.153918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:51.163762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d87cfb6-0692-41e1-aec1-0277fa8ea11d service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:51.230315 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:51.231207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:51.231497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:51.231793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] PATH_INFO: `/v3/projects/cd9af4b7d95e492cad7f4b6a9bebc7c5` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:51.233150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9af4b7d95e492cad7f4b6a9bebc7c5)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:51.235041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:51.324864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9af4b7d95e492cad7f4b6a9bebc7c5', 'request_id': 'req-ae4f0406-efe3-4c5d-9050-153fe0312d79'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:51.331309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9af4b7d95e492cad7f4b6a9bebc7c5', 'request_id': 'req-ae4f0406-efe3-4c5d-9050-153fe0312d79'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:51.337024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9af4b7d95e492cad7f4b6a9bebc7c5', 'request_id': 'req-ae4f0406-efe3-4c5d-9050-153fe0312d79'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:51.562926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] The token cache is being invalidate because project cd9af4b7d95e492cad7f4b6a9bebc7c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:29:51.563601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:51.564104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae4f0406-efe3-4c5d-9050-153fe0312d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:29:51.565687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1169/2343] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:29:51 2022] DELETE /identity/v3/projects/cd9af4b7d95e492cad7f4b6a9bebc7c5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:51.571766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461249c-d964-43a9-b707-63ac917beb64 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:51.572252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461249c-d964-43a9-b707-63ac917beb64 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:51.572471 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461249c-d964-43a9-b707-63ac917beb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:51.594094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8461249c-d964-43a9-b707-63ac917beb64 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:51.601949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8461249c-d964-43a9-b707-63ac917beb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:51.666405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1175/2344] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:51.674896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae15b13-6ae0-433e-b10d-f2fe4d5a87a9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:51.675195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae15b13-6ae0-433e-b10d-f2fe4d5a87a9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:51.675518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae15b13-6ae0-433e-b10d-f2fe4d5a87a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:51.694347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-fae15b13-6ae0-433e-b10d-f2fe4d5a87a9 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:51.695559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-fae15b13-6ae0-433e-b10d-f2fe4d5a87a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:51.703713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1170/2345] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:51.708439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461249c-d964-43a9-b707-63ac917beb64 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:51.719522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461249c-d964-43a9-b707-63ac917beb64 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:51.724220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:51.724871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:51.725150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:51.725424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:51.726787 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:29:51.727096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:51.728742 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:51.751842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1176/2346] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:29:51.752553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae15b13-6ae0-433e-b10d-f2fe4d5a87a9 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:51.761863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae15b13-6ae0-433e-b10d-f2fe4d5a87a9 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:51.766516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:51.767163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:51.767448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:51.767725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:51.768444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:51.771339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:51.874202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1171/2347] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:51 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:51.878273 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:51.888490 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970fa30-04fa-49a4-9308-1bccfd3dd66a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:51.947401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:51.948119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:51.948406 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:51.948686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:51.949391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:51.951321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:52.054946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1177/2348] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:51 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:52.059197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:52.070623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b56858-6f4e-4c3d-a2ee-887042d45ef0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:52.075455 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:52.076083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:52.076374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:52.076660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:52.077375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:52.078960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:52.099779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1172/2349] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:52.117223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:52.118453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bcb47e-d766-454d-9b5c-8e24dcfb363a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:52.123528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0c052-02b4-41e2-9b96-3ec892bf9ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:52.124243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c052-02b4-41e2-9b96-3ec892bf9ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:52.124523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c052-02b4-41e2-9b96-3ec892bf9ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:52.124805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0c052-02b4-41e2-9b96-3ec892bf9ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:52.125537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c052-02b4-41e2-9b96-3ec892bf9ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:52.127180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e0c052-02b4-41e2-9b96-3ec892bf9ae8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:52.141703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1178/2350] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:29:52.145870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:52.156594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2cf87-a7ab-4123-a936-0c7ec5f7c15b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:52.161312 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:52.162199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:52.162724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:52.162835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] PATH_INFO: `/v3/projects/8cc53f5efd5d4d8cacf10dcdc01bb463/users/c3293734bb064e34946e16c72263449a/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:52.178208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cc53f5efd5d4d8cacf10dcdc01bb463, user_id=c3293734bb064e34946e16c72263449a, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:52.181004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:52.262911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1173/2351] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:52 2022] PUT /identity/v3/projects/8cc53f5efd5d4d8cacf10dcdc01bb463/users/c3293734bb064e34946e16c72263449a/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:52.275358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a6769-ba24-49c2-8995-a1738c10d7a8 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:52.277668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a6769-ba24-49c2-8995-a1738c10d7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:52.277668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a6769-ba24-49c2-8995-a1738c10d7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:52.306141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-634a6769-ba24-49c2-8995-a1738c10d7a8 None None] MFA Rules not processed for user `c3293734bb064e34946e16c72263449a`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:52.307355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-634a6769-ba24-49c2-8995-a1738c10d7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:52.357140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:52.377839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1179/2352] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:29:52.390317 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:52.392376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:52.416225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:52.479347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d29f0f0-4dc8-4548-b5fe-3fece2e51835 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:52.486890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:52.488063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:52.488342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:52.488625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:52.500069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:52.501897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:52.516080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1174/2353] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:52 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:29:57.192736 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc6fe-e906-4907-a4a3-11b21da0e0d9 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:57.193245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc6fe-e906-4907-a4a3-11b21da0e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:57.193366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fc6fe-e906-4907-a4a3-11b21da0e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:57.234287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7c7fc6fe-e906-4907-a4a3-11b21da0e0d9 None None] MFA Rules not processed for user `b9537b7add3b4b5da501903db49a2a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:57.237089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7c7fc6fe-e906-4907-a4a3-11b21da0e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:57.337183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1180/2354] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:57.347050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:57.349274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:57.352607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865938f-1aca-4618-b359-7eab39dfa807 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:57.353059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865938f-1aca-4618-b359-7eab39dfa807 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:57.353349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8865938f-1aca-4618-b359-7eab39dfa807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:57.371591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8865938f-1aca-4618-b359-7eab39dfa807 None None] MFA Rules not processed for user `88575c0bd9db4fa8bc9bd511c0ac37dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:29:57.372848 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8865938f-1aca-4618-b359-7eab39dfa807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:57.422011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d648d-a24b-4fb3-9d69-5b2a63ce0ab8 service glance] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:57.426530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:57.427462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:57.427786 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:57.428075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:57.429797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:57.432251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:57.435910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1181/2355] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:29:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:29:57.533575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1175/2356] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:29:57 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:57.537617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865938f-1aca-4618-b359-7eab39dfa807 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:57.555701 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8865938f-1aca-4618-b359-7eab39dfa807 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:57.667172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:57.667899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:57.668189 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:57.668836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:57.669198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:57.671832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:57.813309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1182/2357] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:29:57 2022] POST /identity/v3/users => generated 298 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:29:57.816886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:57.835239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d570b36-8ed0-4019-a104-fb3c013a8f18 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:57.842840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:57.844394 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:57.844796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:57.845152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:57.846143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:57.848000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:57.862724 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1176/2358] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:57.868151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:57.894338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d6034-63b9-4cb5-a10d-beff5a1ef38a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:57.900197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:57.901549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:57.901549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:57.901772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] PATH_INFO: `/v3/projects/17c68dbe9a4a402685d6e60409d6b3a7/users/0d804c83a57244b595447f8400d2fa28/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:57.915181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:57.926805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c68dbe9a4a402685d6e60409d6b3a7, user_id=0d804c83a57244b595447f8400d2fa28, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:57.940109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:58.024860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1183/2359] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:57 2022] PUT /identity/v3/projects/17c68dbe9a4a402685d6e60409d6b3a7/users/0d804c83a57244b595447f8400d2fa28/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:58.028528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:58.060966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc5cef-bba9-4e7f-a159-f7ed1ae3c6f2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:58.178143 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:58.179595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:58.180036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:58.180382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:58.181212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:58.188620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:58.203788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1177/2360] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:29:58 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:29:58.207915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:58.222204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af13ac9-ac73-4410-8af8-b23eb1998eab admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:58.228438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e00e5d-1809-4306-8163-19bca186f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:58.229624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00e5d-1809-4306-8163-19bca186f1d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:58.229912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00e5d-1809-4306-8163-19bca186f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:58.230199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00e5d-1809-4306-8163-19bca186f1d8 admin admin] PATH_INFO: `/v3/projects/17c68dbe9a4a402685d6e60409d6b3a7/users/0d804c83a57244b595447f8400d2fa28/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:58.232646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00e5d-1809-4306-8163-19bca186f1d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17c68dbe9a4a402685d6e60409d6b3a7, user_id=0d804c83a57244b595447f8400d2fa28, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:58.235917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00e5d-1809-4306-8163-19bca186f1d8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:58.250598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1184/2361] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:29:58 2022] PUT /identity/v3/projects/17c68dbe9a4a402685d6e60409d6b3a7/users/0d804c83a57244b595447f8400d2fa28/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:29:58.336121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:29:58.338026 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:58.353931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:29:58.516807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b1bf50-9768-4c7d-aee8-5a479627ea18 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:29:58.660895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:29:58.661983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:29:58.662504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:29:58.662795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:29:58.666648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:29:58.687310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:29:58.688719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:29:58.704036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1178/2362] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:29:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:30:07.392547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdc104-626f-4c8b-8aec-dbe38a680ba8 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:30:07.398491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdc104-626f-4c8b-8aec-dbe38a680ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:30:07.398491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbdc104-626f-4c8b-8aec-dbe38a680ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:30:07.428108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-bbbdc104-626f-4c8b-8aec-dbe38a680ba8 None None] MFA Rules not processed for user `0d804c83a57244b595447f8400d2fa28`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:30:07.429774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bbbdc104-626f-4c8b-8aec-dbe38a680ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:30:07.515210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1185/2363] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:30:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:30:07.535639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:30:07.537096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:30:07.607378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7e0c8-6cad-49dc-b154-585ee92ce798 service placement] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:30:07.612723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:30:07.613884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:30:07.614402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:30:07.616825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:30:07.728738 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:30:07.775198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:30:07.784061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:30:07.794068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1179/2364] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:30:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:30:08.086107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e41305-e9b4-4ea9-8f0e-7fe68aa94b29 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:30:08.086404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e41305-e9b4-4ea9-8f0e-7fe68aa94b29 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:30:08.086683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e41305-e9b4-4ea9-8f0e-7fe68aa94b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:30:08.158400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-79e41305-e9b4-4ea9-8f0e-7fe68aa94b29 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:30:08.159723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-79e41305-e9b4-4ea9-8f0e-7fe68aa94b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:30:08.232232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1186/2365] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:30:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:30:08.285276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:30:08.286857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:30:08.395248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a0129-3102-4239-a9f7-ceca6f7b7b15 service glance] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:30:08.400685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31b112-b5b9-4f5d-a1bd-997f7e610a5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:30:08.401337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b112-b5b9-4f5d-a1bd-997f7e610a5a service swift] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:30:08.401622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b112-b5b9-4f5d-a1bd-997f7e610a5a service swift] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:30:08.401899 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b112-b5b9-4f5d-a1bd-997f7e610a5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:30:08.410401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31b112-b5b9-4f5d-a1bd-997f7e610a5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:30:08.412017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31b112-b5b9-4f5d-a1bd-997f7e610a5a service swift] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:30:08.420241 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1180/2366] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:30:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:30:18.227808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:30:18.416791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=51 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:30:20.446327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e99444-2704-4154-a0a7-2bab603ac26b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:30:20.446924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e99444-2704-4154-a0a7-2bab603ac26b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:30:20.446924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e99444-2704-4154-a0a7-2bab603ac26b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:30:20.468771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d2e99444-2704-4154-a0a7-2bab603ac26b None None] MFA Rules not processed for user `b9537b7add3b4b5da501903db49a2a11`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:30:20.469994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d2e99444-2704-4154-a0a7-2bab603ac26b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:30:20.478109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1187/2367] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:30:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:30:30.461781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:30:37.047349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:30:37.047869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:30:37.048047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:30:37.077910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:30:37.079190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:30:37.096907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1181/2368] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:30:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:30:37.112620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e99444-2704-4154-a0a7-2bab603ac26b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:30:37.131259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e99444-2704-4154-a0a7-2bab603ac26b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:30:37.137094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:30:37.140887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:30:37.140887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:30:37.140887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:30:37.148115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:30:37.150277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:30:37.165989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1188/2369] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:30:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:30:47.066923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:30:47.159134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:07.058269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:07.063491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:07.128822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898fc1a3-e20d-4ce2-aea7-ce859107b84a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:07.133581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e80f9-8227-438b-97ee-65565dabe142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:07.134238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e80f9-8227-438b-97ee-65565dabe142 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:07.134566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e80f9-8227-438b-97ee-65565dabe142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:07.134854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e80f9-8227-438b-97ee-65565dabe142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:07.136727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d64e80f9-8227-438b-97ee-65565dabe142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:07.209388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e80f9-8227-438b-97ee-65565dabe142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:07.210918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e80f9-8227-438b-97ee-65565dabe142 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:07.221748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1182/2370] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:08.107824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:08.109487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:08.178694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c3cb7-3bd1-4911-9367-43dc0b17bd00 service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:08.183201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:08.184207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:08.184498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:08.184777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] PATH_INFO: `/v3/users/33973531257e483bbf0504ed7401d1b2` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:08.186000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=33973531257e483bbf0504ed7401d1b2)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:08.188012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:08.401296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33973531257e483bbf0504ed7401d1b2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:08.415543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33973531257e483bbf0504ed7401d1b2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:08.508722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33973531257e483bbf0504ed7401d1b2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:08.509924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33973531257e483bbf0504ed7401d1b2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:08.510930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33973531257e483bbf0504ed7401d1b2'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:08.531600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1189/2371] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:31:08 2022] DELETE /identity/v3/users/33973531257e483bbf0504ed7401d1b2 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:08.536010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:08.536604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:08.536604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:08.571470 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:08.572894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:08.694770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1183/2372] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:08.715214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:08.721031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:08.800509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d361c-0fd7-4e87-a331-db3d925af99d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:08.809487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5fd9a-c246-4daf-bc9f-4c35ef78478e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:08.809487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5fd9a-c246-4daf-bc9f-4c35ef78478e service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:08.809487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5fd9a-c246-4daf-bc9f-4c35ef78478e service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:08.809487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5fd9a-c246-4daf-bc9f-4c35ef78478e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:08.816855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5fd9a-c246-4daf-bc9f-4c35ef78478e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:08.818312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5fd9a-c246-4daf-bc9f-4c35ef78478e service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:08.829094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1190/2373] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:10.082240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:10.083848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:10.108986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fa63b2-2018-40bf-99d0-3e6f6c7e246f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:10.114709 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:10.115725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.121431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.121431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] PATH_INFO: `/v3/projects/e8d00b49a01c4abebca9be6489a152c3` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.121431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8d00b49a01c4abebca9be6489a152c3)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:10.121431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:10.269642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:10.272425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8d00b49a01c4abebca9be6489a152c3', 'request_id': 'req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:10.279481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8d00b49a01c4abebca9be6489a152c3', 'request_id': 'req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:10.285184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8d00b49a01c4abebca9be6489a152c3', 'request_id': 'req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:10.379019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] The token cache is being invalidate because project e8d00b49a01c4abebca9be6489a152c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:31:10.379239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:10.379936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f490d033-4ec0-4e75-a6f7-3de4bc5fa467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:10.381743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1184/2374] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:31:10 2022] DELETE /identity/v3/projects/e8d00b49a01c4abebca9be6489a152c3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:10.387053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a91b00-d0b2-4b34-b4e8-488d743c11a0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.387590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a91b00-d0b2-4b34-b4e8-488d743c11a0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.387590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a91b00-d0b2-4b34-b4e8-488d743c11a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.405791 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d1a91b00-d0b2-4b34-b4e8-488d743c11a0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:10.407091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d1a91b00-d0b2-4b34-b4e8-488d743c11a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:10.427700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:10.479197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1191/2375] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:10.487511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6ab6-3830-449a-8126-6549d1129ec1 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.488149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6ab6-3830-449a-8126-6549d1129ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.490328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7e6ab6-3830-449a-8126-6549d1129ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.514667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2e7e6ab6-3830-449a-8126-6549d1129ec1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:10.515926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2e7e6ab6-3830-449a-8126-6549d1129ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:10.525362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1185/2376] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:10.533541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a91b00-d0b2-4b34-b4e8-488d743c11a0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:10.544102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a91b00-d0b2-4b34-b4e8-488d743c11a0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:10.549383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:10.550064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.550348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.550625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.551673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:31:10.552018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:10.553660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:10.571909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1192/2377] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:31:10.575603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6ab6-3830-449a-8126-6549d1129ec1 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:10.586590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7e6ab6-3830-449a-8126-6549d1129ec1 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:10.591948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:10.593210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.593210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.593210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.593863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:10.595837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:10.719247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1186/2378] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:31:10 2022] POST /identity/v3/projects => generated 399 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:31:10.723202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:10.736251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9db9d8-57d1-4609-a198-63993a8727dc admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:10.810854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:10.811557 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.811890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.812232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.813223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:10.815218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:10.911571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1193/2379] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:31:10 2022] POST /identity/v3/users => generated 300 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:31:10.911895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:10.925613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8507d7-07d3-430a-a78f-daff336a4eb9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:10.935154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:10.936717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.937069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.937418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.938298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:10.941620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:10.956021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1187/2380] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:10.960794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:10.975202 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55df9f-a1d0-4e6c-8ff6-d52df2079ca6 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:10.981986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f90c01-f31b-458d-af2f-55945c350d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:10.982910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90c01-f31b-458d-af2f-55945c350d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:10.983350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90c01-f31b-458d-af2f-55945c350d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:10.983865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f90c01-f31b-458d-af2f-55945c350d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:10.985620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f90c01-f31b-458d-af2f-55945c350d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:10.987232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f90c01-f31b-458d-af2f-55945c350d1f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:11.002052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1194/2381] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:31:11.006418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:11.018712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4a524f-1ec0-4e7c-b3af-87833d7c629d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:11.024623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:11.025629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:11.034515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:11.034515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] PATH_INFO: `/v3/projects/e932939419ce4d52a0c222408ca31a87/users/f3dcb871043e42b6a9adb164cc12ba54/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:11.047310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e932939419ce4d52a0c222408ca31a87, user_id=f3dcb871043e42b6a9adb164cc12ba54, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:11.051647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:11.128374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1188/2382] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:31:11 2022] PUT /identity/v3/projects/e932939419ce4d52a0c222408ca31a87/users/f3dcb871043e42b6a9adb164cc12ba54/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:31:11.139677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bff355-39bf-47b8-ad84-2fc327f5d56e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:11.140281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bff355-39bf-47b8-ad84-2fc327f5d56e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:11.140663 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bff355-39bf-47b8-ad84-2fc327f5d56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:11.170408 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-53bff355-39bf-47b8-ad84-2fc327f5d56e None None] MFA Rules not processed for user `f3dcb871043e42b6a9adb164cc12ba54`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:11.171552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-53bff355-39bf-47b8-ad84-2fc327f5d56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:11.241658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1195/2383] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:11.253864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:11.255616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:11.315025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7327a91-c703-4d1c-b02c-3d0df51b404a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:11.318889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:11.319627 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:11.320003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:11.320387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:11.329159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:11.330600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:11.340100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1189/2384] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:21.239070 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:21.335509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:32.769623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc235ac-62b9-4ec2-80ad-d8110ca2f433 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:32.770140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc235ac-62b9-4ec2-80ad-d8110ca2f433 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:32.770236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc235ac-62b9-4ec2-80ad-d8110ca2f433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:32.789254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-cfc235ac-62b9-4ec2-80ad-d8110ca2f433 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:32.790552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cfc235ac-62b9-4ec2-80ad-d8110ca2f433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:32.859729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1196/2385] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:31:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:32.885897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:32.887396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:32.972401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9da7e-3338-4e31-9186-7356d27f4645 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:32.977453 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82720af1-bc0c-4900-a7ac-de93ad9737bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:32.978105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82720af1-bc0c-4900-a7ac-de93ad9737bd service swift] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:32.978391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82720af1-bc0c-4900-a7ac-de93ad9737bd service swift] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:32.978670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82720af1-bc0c-4900-a7ac-de93ad9737bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:32.988171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82720af1-bc0c-4900-a7ac-de93ad9737bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:32.989772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82720af1-bc0c-4900-a7ac-de93ad9737bd service swift] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:32.998947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1190/2386] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:31:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:31:33.221962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc235ac-62b9-4ec2-80ad-d8110ca2f433 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:33.294629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc235ac-62b9-4ec2-80ad-d8110ca2f433 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:33.304609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:33.305407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:33.305695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:33.305971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] PATH_INFO: `/v3/users/f3dcb871043e42b6a9adb164cc12ba54` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:33.307237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3dcb871043e42b6a9adb164cc12ba54)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:33.320425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:34.009173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3dcb871043e42b6a9adb164cc12ba54'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:34.034834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3dcb871043e42b6a9adb164cc12ba54'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:34.150737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3dcb871043e42b6a9adb164cc12ba54'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:34.151523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3dcb871043e42b6a9adb164cc12ba54'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:34.152262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3dcb871043e42b6a9adb164cc12ba54'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:34.167330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1197/2387] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:31:33 2022] DELETE /identity/v3/users/f3dcb871043e42b6a9adb164cc12ba54 => generated 0 bytes in 946 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:34.171807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:34.172372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:34.172372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:34.195790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:34.197439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:34.278429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1191/2388] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:34.288778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:34.290211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:34.322363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:34.359489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa86494-714a-4873-82a1-ef7090adb25e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:34.363347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0a8ea-5044-4da1-9750-dc2c7ccb234a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:34.368677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0a8ea-5044-4da1-9750-dc2c7ccb234a service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:34.368677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0a8ea-5044-4da1-9750-dc2c7ccb234a service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:34.368677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0a8ea-5044-4da1-9750-dc2c7ccb234a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:34.374810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0a8ea-5044-4da1-9750-dc2c7ccb234a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:34.376354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0a8ea-5044-4da1-9750-dc2c7ccb234a service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:34.392465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1198/2389] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:35.006224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:35.008559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:35.051787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e08b3-1d4b-4943-94f7-770425e400d5 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:35.069960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:35.073537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.074133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.075774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] PATH_INFO: `/v3/projects/e932939419ce4d52a0c222408ca31a87` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.079092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e932939419ce4d52a0c222408ca31a87)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:35.090095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:35.216562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e932939419ce4d52a0c222408ca31a87', 'request_id': 'req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:35.223625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e932939419ce4d52a0c222408ca31a87', 'request_id': 'req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:35.230116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e932939419ce4d52a0c222408ca31a87', 'request_id': 'req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:35.287157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] The token cache is being invalidate because project e932939419ce4d52a0c222408ca31a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:31:35.287594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:35.288436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9b5b6697-a80f-4d75-8e5f-7cf74de8b2fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:35.290533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1192/2390] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:31:35 2022] DELETE /identity/v3/projects/e932939419ce4d52a0c222408ca31a87 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:35.296513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d8af-1bec-425f-82da-58d0aeb58458 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.297720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d8af-1bec-425f-82da-58d0aeb58458 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.298067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e7d8af-1bec-425f-82da-58d0aeb58458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.322626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-c0e7d8af-1bec-425f-82da-58d0aeb58458 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:35.323955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c0e7d8af-1bec-425f-82da-58d0aeb58458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:35.400380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1199/2391] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:31:35.409239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb924b-34fd-42ba-9d22-24c66395f54d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.409605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb924b-34fd-42ba-9d22-24c66395f54d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.409960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb924b-34fd-42ba-9d22-24c66395f54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.438428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-cafb924b-34fd-42ba-9d22-24c66395f54d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:35.439868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-cafb924b-34fd-42ba-9d22-24c66395f54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:35.450648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1193/2392] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:35.457161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7d8af-1bec-425f-82da-58d0aeb58458 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:35.474344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e7d8af-1bec-425f-82da-58d0aeb58458 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:35.485962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:35.485962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.485962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.486405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.487154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:31:35.487460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:35.489140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:35.504097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1200/2393] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:31:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:31:35.507398 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb924b-34fd-42ba-9d22-24c66395f54d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:35.518295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb924b-34fd-42ba-9d22-24c66395f54d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:35.526409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:35.527334 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.527822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.528863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.529787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:35.531907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:35.535715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:35.657712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1194/2394] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:31:35 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:31:35.658200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:35.671172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c62d8c2-1ad1-4a21-85f5-37211363e21e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:35.750925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:35.751511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.751818 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.752095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.752804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:35.754966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:35.843175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1201/2395] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:31:35 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:31:35.847989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:35.861569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94289e0-9ccf-4008-8fc7-c992ccd57c3a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:35.868144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:35.869276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.869276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.869480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.870354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:35.872719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:35.895860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1195/2396] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:35.896092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:35.909297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96016516-0b7c-4a8f-a154-b4d2510ca257 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:35.914247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c38b7-a093-4fe9-a06b-11f2bc60807e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:35.914943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c38b7-a093-4fe9-a06b-11f2bc60807e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.915225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c38b7-a093-4fe9-a06b-11f2bc60807e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.915503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c38b7-a093-4fe9-a06b-11f2bc60807e admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.916286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c38b7-a093-4fe9-a06b-11f2bc60807e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:35.917973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c38b7-a093-4fe9-a06b-11f2bc60807e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:35.933874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1202/2397] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:35.936005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:35.949041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c2216-6b2f-4545-8e9a-23c6210b9361 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:35.954601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:35.955614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:35.955928 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:35.956204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] PATH_INFO: `/v3/projects/77c3cd3a54424befb4b424e6de1f7870/users/38166757db224367bb8208f6366a4184/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:35.974646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77c3cd3a54424befb4b424e6de1f7870, user_id=38166757db224367bb8208f6366a4184, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:35.982280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:36.062188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1196/2398] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:31:35 2022] PUT /identity/v3/projects/77c3cd3a54424befb4b424e6de1f7870/users/38166757db224367bb8208f6366a4184/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:36.081407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8b51c-4619-4e57-a4a2-f8918f2393eb None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:36.081701 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8b51c-4619-4e57-a4a2-f8918f2393eb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:36.081981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8b51c-4619-4e57-a4a2-f8918f2393eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:36.105893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-75d8b51c-4619-4e57-a4a2-f8918f2393eb None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:36.108439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-75d8b51c-4619-4e57-a4a2-f8918f2393eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:36.155258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:36.197142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1203/2399] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:36.211222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:36.233163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:36.284721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b606c-1b1c-4177-8a08-97851b01be08 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:36.288476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:36.289130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:36.289408 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:36.289685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:36.299518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:36.301281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:36.311704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1197/2400] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:31:39.656402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9570f-7f34-4d90-ae21-a994a0ddc8bd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:39.657001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9570f-7f34-4d90-ae21-a994a0ddc8bd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:39.657001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9570f-7f34-4d90-ae21-a994a0ddc8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:39.715954 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-81a9570f-7f34-4d90-ae21-a994a0ddc8bd None None] MFA Rules not processed for user `6a658b1951a9426d94341940d87b5f95`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:39.717209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-81a9570f-7f34-4d90-ae21-a994a0ddc8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:39.816779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1204/2401] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:40.854124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:40.856458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:40.879274 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fba19d-5426-44a9-bb52-77c9ab85555b service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:40.892652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:40.896219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:40.896661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:40.897106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] PATH_INFO: `/v3/users/c3293734bb064e34946e16c72263449a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:40.900164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3293734bb064e34946e16c72263449a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:40.902294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:41.173721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3293734bb064e34946e16c72263449a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:41.210660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3293734bb064e34946e16c72263449a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:41.309793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3293734bb064e34946e16c72263449a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:41.310734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3293734bb064e34946e16c72263449a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:41.311934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3293734bb064e34946e16c72263449a'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:41.350934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1198/2402] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:31:40 2022] DELETE /identity/v3/users/c3293734bb064e34946e16c72263449a => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:41.355244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:41.355835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:41.355835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:41.390128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:41.391410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:41.509642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1205/2403] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:41.520139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:41.522592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:41.613604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:41.677635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21cc6e-8ba7-46d4-b7fa-3168ae1b6d89 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:41.687123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e935486-c131-4fdb-9362-cfb9d65b2aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:41.688558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e935486-c131-4fdb-9362-cfb9d65b2aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:41.690027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e935486-c131-4fdb-9362-cfb9d65b2aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:41.690585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e935486-c131-4fdb-9362-cfb9d65b2aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:41.717278 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e935486-c131-4fdb-9362-cfb9d65b2aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:41.718749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e935486-c131-4fdb-9362-cfb9d65b2aa6 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:41.741837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1199/2404] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:42.605294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:42.608489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:42.669366 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f05db6-48a2-473f-837c-67f90c18dac6 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:42.722467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:42.722467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:42.722467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:42.722467 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] PATH_INFO: `/v3/projects/8cc53f5efd5d4d8cacf10dcdc01bb463` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:42.728604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cc53f5efd5d4d8cacf10dcdc01bb463)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:42.730659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:42.912862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cc53f5efd5d4d8cacf10dcdc01bb463', 'request_id': 'req-abcd6b83-8af1-4a70-8224-53de1083de09'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:42.926447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cc53f5efd5d4d8cacf10dcdc01bb463', 'request_id': 'req-abcd6b83-8af1-4a70-8224-53de1083de09'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:42.937941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cc53f5efd5d4d8cacf10dcdc01bb463', 'request_id': 'req-abcd6b83-8af1-4a70-8224-53de1083de09'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:43.016170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] The token cache is being invalidate because project 8cc53f5efd5d4d8cacf10dcdc01bb463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:31:43.017373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:43.019456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-abcd6b83-8af1-4a70-8224-53de1083de09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:43.022331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1206/2405] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:31:42 2022] DELETE /identity/v3/projects/8cc53f5efd5d4d8cacf10dcdc01bb463 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:43.027605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbe21-9e00-4780-ac17-006d9f7955c4 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:43.027605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbe21-9e00-4780-ac17-006d9f7955c4 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:43.028796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbe21-9e00-4780-ac17-006d9f7955c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:43.076308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b7cbbe21-9e00-4780-ac17-006d9f7955c4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:43.076308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b7cbbe21-9e00-4780-ac17-006d9f7955c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:43.246051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1200/2406] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:43.253588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aec49-4ff6-4445-93a6-e416b4d07848 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:43.254357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aec49-4ff6-4445-93a6-e416b4d07848 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:43.254777 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aec49-4ff6-4445-93a6-e416b4d07848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:43.263987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:43.281864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-676aec49-4ff6-4445-93a6-e416b4d07848 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:43.283568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-676aec49-4ff6-4445-93a6-e416b4d07848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:43.302844 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1207/2407] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:43.307717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbe21-9e00-4780-ac17-006d9f7955c4 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:43.333816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbbe21-9e00-4780-ac17-006d9f7955c4 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:43.339579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:43.340347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:43.343854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:43.343854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:43.343854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:31:43.343854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:43.344702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:43.388529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1201/2408] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:31:43.394216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aec49-4ff6-4445-93a6-e416b4d07848 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:43.406505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aec49-4ff6-4445-93a6-e416b4d07848 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:43.428565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:43.429648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:43.430210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:43.430691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:43.431670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:43.435950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:43.612384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1208/2409] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:31:43 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:31:43.618186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:43.644365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cc878-e720-4077-a5e5-21534c7a7316 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:43.777614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:43.779293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:43.779840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:43.780289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:43.782374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:43.786364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:43.901053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1202/2410] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:31:43 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:31:43.901927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:43.917942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7884623-1c63-4663-a8cd-3da6db3aaee5 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:43.926992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:43.927995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:43.928469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:43.928793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:43.929540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:43.931619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:43.952530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1209/2411] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:43.959082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:44.030335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b48ca-d6f9-4083-9b8d-be03688d3c29 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:44.060713 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf1df98-8fe5-4aa9-8753-1ab6850601d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:44.061333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1df98-8fe5-4aa9-8753-1ab6850601d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:44.061617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1df98-8fe5-4aa9-8753-1ab6850601d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:44.061893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf1df98-8fe5-4aa9-8753-1ab6850601d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:44.062671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1df98-8fe5-4aa9-8753-1ab6850601d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:44.064430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf1df98-8fe5-4aa9-8753-1ab6850601d8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:44.090810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1203/2412] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:44.092765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:44.117581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698a53-a6e3-4f89-9129-6eee326b642e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:44.130300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:44.131445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:44.131855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:44.132230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] PATH_INFO: `/v3/projects/be729e11723047f1905a3ae72f359a30/users/29cd4af5cc514da191cdb9dba9aade08/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:44.161935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be729e11723047f1905a3ae72f359a30, user_id=29cd4af5cc514da191cdb9dba9aade08, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:44.165101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:44.248917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1210/2413] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:31:44 2022] PUT /identity/v3/projects/be729e11723047f1905a3ae72f359a30/users/29cd4af5cc514da191cdb9dba9aade08/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:44.259737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21761e3f-1c49-4d16-a79a-024e6c5e9ab7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:44.260220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21761e3f-1c49-4d16-a79a-024e6c5e9ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:44.260807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21761e3f-1c49-4d16-a79a-024e6c5e9ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:44.321365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-21761e3f-1c49-4d16-a79a-024e6c5e9ab7 None None] MFA Rules not processed for user `29cd4af5cc514da191cdb9dba9aade08`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:44.322996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-21761e3f-1c49-4d16-a79a-024e6c5e9ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:44.382045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:44.473633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1204/2414] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:44.486522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:44.488583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:44.643508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ec20f-23f0-407b-af41-147f3acbc8e6 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:44.653985 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae82c5c-eba2-41ab-986a-bd73c6846219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:44.654916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82c5c-eba2-41ab-986a-bd73c6846219 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:44.655377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82c5c-eba2-41ab-986a-bd73c6846219 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:44.655847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae82c5c-eba2-41ab-986a-bd73c6846219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:44.680424 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae82c5c-eba2-41ab-986a-bd73c6846219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:44.681890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae82c5c-eba2-41ab-986a-bd73c6846219 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:44.725426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1211/2415] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:45.821305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:45.821754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:45.821754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:45.853240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] MFA Rules not processed for user `6214705ae31e459891f7a215f45a4cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:45.854616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:45.951617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1205/2416] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:46.697386 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286e4c1-6d7f-4062-be7a-8c721a7d1358 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:46.698284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286e4c1-6d7f-4062-be7a-8c721a7d1358 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:46.698749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286e4c1-6d7f-4062-be7a-8c721a7d1358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:46.735463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e286e4c1-6d7f-4062-be7a-8c721a7d1358 None None] MFA Rules not processed for user `38166757db224367bb8208f6366a4184`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:46.736437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e286e4c1-6d7f-4062-be7a-8c721a7d1358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:46.854000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1212/2417] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:46.859336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:46.860873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:46.944137 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b2470-d52c-4203-b6a6-33fb562e32db None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:46.952656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:46.953338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:46.953806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:46.954081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:46.955831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:46.957735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:47.146997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1206/2418] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:31:46 2022] POST /identity/v3/projects => generated 373 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:31:47.151254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286e4c1-6d7f-4062-be7a-8c721a7d1358 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:47.163554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286e4c1-6d7f-4062-be7a-8c721a7d1358 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:47.216277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:47.317371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:47.320385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:47.325841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:47.325841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:47.325841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:47.334355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:47.482778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1213/2419] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:31:47 2022] POST /identity/v3/users => generated 293 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:31:47.482890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:47.504285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e45b6ec-2f80-41f1-a9f4-600fd53aceec admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:47.529038 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:47.533162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:47.534650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:47.535718 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:47.538449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:47.544119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:47.580166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1207/2420] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:31:47.582124 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:47.597316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5244ad-e6b9-4c8d-bf8c-d95d8ae9e331 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:47.614045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:47.615215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:47.615596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:47.616105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] PATH_INFO: `/v3/projects/7ddb778984b64029ae40ad6be7f1bf20/users/5316267bacc147ed9fa67e0a9dc16338/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:47.637215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ddb778984b64029ae40ad6be7f1bf20, user_id=5316267bacc147ed9fa67e0a9dc16338, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:47.640428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:47.822790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1214/2421] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:31:47 2022] PUT /identity/v3/projects/7ddb778984b64029ae40ad6be7f1bf20/users/5316267bacc147ed9fa67e0a9dc16338/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:47.827325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:47.853004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8a0e0-7666-49b2-99fa-f6cbb76f3a25 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:47.980003 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:48.033880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:48.034571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:48.034860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:48.035145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:48.036059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:48.037678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:48.060694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1208/2422] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:48.065010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:48.102220 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ef423-45f5-4855-be82-3c8290df3538 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:48.113402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb4c32-0e9d-48ac-8b20-bc0247e2bd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:48.114277 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb4c32-0e9d-48ac-8b20-bc0247e2bd39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:48.118619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb4c32-0e9d-48ac-8b20-bc0247e2bd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:48.119256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb4c32-0e9d-48ac-8b20-bc0247e2bd39 admin admin] PATH_INFO: `/v3/projects/7ddb778984b64029ae40ad6be7f1bf20/users/5316267bacc147ed9fa67e0a9dc16338/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:48.129683 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb4c32-0e9d-48ac-8b20-bc0247e2bd39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ddb778984b64029ae40ad6be7f1bf20, user_id=5316267bacc147ed9fa67e0a9dc16338, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:48.132542 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb4c32-0e9d-48ac-8b20-bc0247e2bd39 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:48.192198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1215/2423] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:31:48 2022] PUT /identity/v3/projects/7ddb778984b64029ae40ad6be7f1bf20/users/5316267bacc147ed9fa67e0a9dc16338/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:31:49.645076 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:49.683560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd463420-cea8-4a3b-a11b-604152be72b2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:49.796603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:49.797508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:49.797799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:49.798133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] PATH_INFO: `/v3/users/29cd4af5cc514da191cdb9dba9aade08` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:49.799629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29cd4af5cc514da191cdb9dba9aade08)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:49.802365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:50.062644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29cd4af5cc514da191cdb9dba9aade08'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:50.092107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29cd4af5cc514da191cdb9dba9aade08'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:50.195478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29cd4af5cc514da191cdb9dba9aade08'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:50.196135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29cd4af5cc514da191cdb9dba9aade08'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:50.196774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29cd4af5cc514da191cdb9dba9aade08'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:50.216336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1209/2424] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:31:49 2022] DELETE /identity/v3/users/29cd4af5cc514da191cdb9dba9aade08 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:50.221340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ad234-265f-40b7-8717-82adae4710cc None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:50.222270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ad234-265f-40b7-8717-82adae4710cc None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:50.222723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ad234-265f-40b7-8717-82adae4710cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:50.248812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-130ad234-265f-40b7-8717-82adae4710cc None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:50.250723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-130ad234-265f-40b7-8717-82adae4710cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:50.345879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1216/2425] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:50.360648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:50.368754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:50.473756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc991a-83c3-45b7-bf0c-efb2ea4c2381 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:50.477850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:50.478522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:50.478859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:50.479300 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:50.496895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:50.499528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:50.508155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:50.520483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1210/2426] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:50.904824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ad234-265f-40b7-8717-82adae4710cc None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:50.906462 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-130ad234-265f-40b7-8717-82adae4710cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:50.930195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ad234-265f-40b7-8717-82adae4710cc None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:50.935242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ca9bc-e8de-422d-8760-73ca77847fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:50.936019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ca9bc-e8de-422d-8760-73ca77847fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:50.936310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ca9bc-e8de-422d-8760-73ca77847fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:50.936587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ca9bc-e8de-422d-8760-73ca77847fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:50.939314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5c7ca9bc-e8de-422d-8760-73ca77847fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:51.083278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ca9bc-e8de-422d-8760-73ca77847fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:51.085058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ca9bc-e8de-422d-8760-73ca77847fe1 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:51.104698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1217/2427] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:52.345935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:52.347443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:52.364561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae1058-82bf-47e3-8b89-1ff7d849f804 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:52.370982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:52.372613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:52.373054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:52.373481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] PATH_INFO: `/v3/projects/be729e11723047f1905a3ae72f359a30` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:52.374924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be729e11723047f1905a3ae72f359a30)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:52.377101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:52.592067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be729e11723047f1905a3ae72f359a30', 'request_id': 'req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:52.601205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be729e11723047f1905a3ae72f359a30', 'request_id': 'req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:52.611329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be729e11723047f1905a3ae72f359a30', 'request_id': 'req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:52.709233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] The token cache is being invalidate because project be729e11723047f1905a3ae72f359a30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:31:52.709906 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:52.713610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b7c5d6ef-41e2-4280-9dc9-e8d6a844ebf2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:31:52.718347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1211/2428] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:31:52 2022] DELETE /identity/v3/projects/be729e11723047f1905a3ae72f359a30 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:52.725005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0c8fd-128b-4fa1-8108-a89fa6f0b4e2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:52.725645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0c8fd-128b-4fa1-8108-a89fa6f0b4e2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:52.725645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0c8fd-128b-4fa1-8108-a89fa6f0b4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:52.756803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e5a0c8fd-128b-4fa1-8108-a89fa6f0b4e2 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:52.758037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e5a0c8fd-128b-4fa1-8108-a89fa6f0b4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:52.790640 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:52.908906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1218/2429] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:52.917024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b1af2-2b9b-4951-b3fe-eea46acabe50 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:52.920708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b1af2-2b9b-4951-b3fe-eea46acabe50 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:52.920708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b1af2-2b9b-4951-b3fe-eea46acabe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:52.940998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c95b1af2-2b9b-4951-b3fe-eea46acabe50 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:52.942557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c95b1af2-2b9b-4951-b3fe-eea46acabe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:52.957561 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1212/2430] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:52.963183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0c8fd-128b-4fa1-8108-a89fa6f0b4e2 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:52.985440 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0c8fd-128b-4fa1-8108-a89fa6f0b4e2 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:52.995703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:52.998185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:52.998585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:52.999870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.000935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:31:53.001873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:53.005116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:53.026103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1219/2431] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:31:53.031186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b1af2-2b9b-4951-b3fe-eea46acabe50 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:53.050588 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b1af2-2b9b-4951-b3fe-eea46acabe50 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:53.060247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:53.064181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:53.064181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:53.064181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.064181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:53.065756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:53.154350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1213/2432] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:31:53 2022] POST /identity/v3/projects => generated 391 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:31:53.158228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:53.176893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe568504-cfa2-4b93-8f43-44b1429c2c2a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:53.312794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:53.313746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:53.314243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:53.314702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.315592 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:53.317959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:53.441228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1220/2433] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:31:53 2022] POST /identity/v3/users => generated 296 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:31:53.442122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:53.453095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7229522-0801-4861-ad5e-b7ee9fd365b2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:53.458464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:53.459501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:53.460036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:53.461945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.461945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:53.463287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:53.476425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1214/2434] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:53.480212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:53.495175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e4dd2-4772-4eef-9fe5-99822474c183 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:53.501363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9c0b-8691-4d6c-a012-6355739ad2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:53.502113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9c0b-8691-4d6c-a012-6355739ad2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:53.502404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9c0b-8691-4d6c-a012-6355739ad2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:53.502684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9c0b-8691-4d6c-a012-6355739ad2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.503792 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa9c0b-8691-4d6c-a012-6355739ad2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:53.505470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa9c0b-8691-4d6c-a012-6355739ad2b8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:53.521768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1221/2435] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:31:53.526604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:53.539537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5523072-723d-4a17-8968-9dd109a1c636 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:53.547691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:53.549212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:53.549714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:53.550218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] PATH_INFO: `/v3/projects/954ae0a9b0c64098ae7137d6a63bdb7c/users/252eb4f862204dd6aca56a7f828b5573/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.570083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=954ae0a9b0c64098ae7137d6a63bdb7c, user_id=252eb4f862204dd6aca56a7f828b5573, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:53.573449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:53.648668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1215/2436] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:31:53 2022] PUT /identity/v3/projects/954ae0a9b0c64098ae7137d6a63bdb7c/users/252eb4f862204dd6aca56a7f828b5573/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:31:53.659855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:53.659855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:53.659855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.695468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] MFA Rules not processed for user `252eb4f862204dd6aca56a7f828b5573`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:53.695468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:53.848394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1222/2437] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:53.855522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:53.858711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:53.932080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86354fa-7722-4484-ae8a-3102aaeb73b6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:53.937952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:53.938956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:53.939485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:53.940020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:53.956332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:53.961437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:53.979147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1216/2438] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:55.242315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:55.243874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:55.275330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:55.347456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da4e639-6f70-4b85-a45d-88b40c341eb3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:55.351545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d29309-a387-4b45-99ca-d9e5c35aa8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:55.352555 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d29309-a387-4b45-99ca-d9e5c35aa8fe service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:55.353003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d29309-a387-4b45-99ca-d9e5c35aa8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:55.353590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d29309-a387-4b45-99ca-d9e5c35aa8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:55.356020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a8d29309-a387-4b45-99ca-d9e5c35aa8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:55.528306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d29309-a387-4b45-99ca-d9e5c35aa8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:55.530238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d29309-a387-4b45-99ca-d9e5c35aa8fe service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:55.560316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1223/2439] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:57.042764 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:57.045527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:57.067907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc0261-cae2-47de-9bdb-b3fcf98d212d service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:57.078181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:57.079341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:57.079639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:57.080170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:57.082651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:57.116957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:57.118871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:57.144035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1217/2440] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:31:57.805490 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876d71-4c39-4de9-87d6-67c8cd19f3ed None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:57.805979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876d71-4c39-4de9-87d6-67c8cd19f3ed None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:57.806116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51876d71-4c39-4de9-87d6-67c8cd19f3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:57.852113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-51876d71-4c39-4de9-87d6-67c8cd19f3ed None None] MFA Rules not processed for user `5316267bacc147ed9fa67e0a9dc16338`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:31:57.853743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-51876d71-4c39-4de9-87d6-67c8cd19f3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:58.017688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1224/2441] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:31:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:31:58.036650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:31:58.038339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:58.068814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd91c28-c5d4-425d-8ce2-763269469466 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:31:58.080230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f622b-fba4-489d-8115-b315607120b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:31:58.082611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f622b-fba4-489d-8115-b315607120b8 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:31:58.082904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f622b-fba4-489d-8115-b315607120b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:31:58.083230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f622b-fba4-489d-8115-b315607120b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:31:58.088548 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1d9f622b-fba4-489d-8115-b315607120b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:31:58.101150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:31:58.197762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f622b-fba4-489d-8115-b315607120b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:31:58.198797 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f622b-fba4-489d-8115-b315607120b8 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:31:58.221471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1218/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:31:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:00.319892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51876d71-4c39-4de9-87d6-67c8cd19f3ed None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:00.331916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51876d71-4c39-4de9-87d6-67c8cd19f3ed None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:00.336022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2944259f-702e-4b0e-89c0-2b17fee8ca42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:00.336919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944259f-702e-4b0e-89c0-2b17fee8ca42 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:00.337676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944259f-702e-4b0e-89c0-2b17fee8ca42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:00.337676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2944259f-702e-4b0e-89c0-2b17fee8ca42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:00.339539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2944259f-702e-4b0e-89c0-2b17fee8ca42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:00.351699 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944259f-702e-4b0e-89c0-2b17fee8ca42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:00.353310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2944259f-702e-4b0e-89c0-2b17fee8ca42 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:00.363960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1225/2443] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:32:00.777518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bc53b-e922-4389-a66a-6a1f76311109 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:00.778029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bc53b-e922-4389-a66a-6a1f76311109 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:00.778224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189bc53b-e922-4389-a66a-6a1f76311109 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:00.782511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1219/2444] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:32:00.790422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848013cb-cf89-4955-b903-2439d2a51772 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:00.790810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848013cb-cf89-4955-b903-2439d2a51772 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:00.790996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848013cb-cf89-4955-b903-2439d2a51772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:00.900709 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-848013cb-cf89-4955-b903-2439d2a51772 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:32:00.902138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-848013cb-cf89-4955-b903-2439d2a51772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:00.928734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1226/2445] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:32:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:32:00.942939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bc53b-e922-4389-a66a-6a1f76311109 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:00.944869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-189bc53b-e922-4389-a66a-6a1f76311109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:01.019712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189bc53b-e922-4389-a66a-6a1f76311109 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:01.023784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7b53e-493e-4878-ac7c-c8b6ae0b688c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:01.032688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b53e-493e-4878-ac7c-c8b6ae0b688c service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.032688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b53e-493e-4878-ac7c-c8b6ae0b688c service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.032688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b53e-493e-4878-ac7c-c8b6ae0b688c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.036895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7b53e-493e-4878-ac7c-c8b6ae0b688c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:01.038704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7b53e-493e-4878-ac7c-c8b6ae0b688c service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:01.051677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1220/2446] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:01.298979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b82ff9-6b8a-4860-ab17-0629fe44e06f None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.299489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b82ff9-6b8a-4860-ab17-0629fe44e06f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.299988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b82ff9-6b8a-4860-ab17-0629fe44e06f None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.301847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1227/2447] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:32:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:32:01.307127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43be1c-e841-476b-8f55-d80254b641e4 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.307418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43be1c-e841-476b-8f55-d80254b641e4 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.307696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43be1c-e841-476b-8f55-d80254b641e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.329701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2b43be1c-e841-476b-8f55-d80254b641e4 None None] MFA Rules not processed for user `e757ff94b7474fd99a1c7221be19c64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:32:01.330930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2b43be1c-e841-476b-8f55-d80254b641e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:01.338606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1221/2448] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:32:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:32:01.352041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b82ff9-6b8a-4860-ab17-0629fe44e06f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:01.369191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b82ff9-6b8a-4860-ab17-0629fe44e06f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:01.376737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d85751-82c2-40b1-9807-53adddb95601 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:01.377661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85751-82c2-40b1-9807-53adddb95601 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.378098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85751-82c2-40b1-9807-53adddb95601 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.383781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d85751-82c2-40b1-9807-53adddb95601 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.395861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:32:01.410340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d85751-82c2-40b1-9807-53adddb95601 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:01.411938 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d85751-82c2-40b1-9807-53adddb95601 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:01.421741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1228/2449] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:01.516599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0fdad-7539-4933-9cc5-223fff3068b5 None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.516913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0fdad-7539-4933-9cc5-223fff3068b5 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.517190 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d0fdad-7539-4933-9cc5-223fff3068b5 None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.518569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1222/2450] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 21:32:01.524758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551950a1-a628-4747-a39d-8a03db42ac9b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.525073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551950a1-a628-4747-a39d-8a03db42ac9b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.525373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551950a1-a628-4747-a39d-8a03db42ac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.545984 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-551950a1-a628-4747-a39d-8a03db42ac9b None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:32:01.547422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-551950a1-a628-4747-a39d-8a03db42ac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:01.559270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1229/2451] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:32:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:32:01.570206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0fdad-7539-4933-9cc5-223fff3068b5 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:01.581821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d0fdad-7539-4933-9cc5-223fff3068b5 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:01.585520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:01.586335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.586641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.586929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.605087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:01.605087 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:01.609081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72759cf-750e-4f9a-a615-1f71373bc7eb None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.609081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72759cf-750e-4f9a-a615-1f71373bc7eb None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.609081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72759cf-750e-4f9a-a615-1f71373bc7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.614731 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1223/2452] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:01.627865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d72759cf-750e-4f9a-a615-1f71373bc7eb None None] MFA Rules not processed for user `38166757db224367bb8208f6366a4184`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:32:01.629092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d72759cf-750e-4f9a-a615-1f71373bc7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:01.636811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1230/2453] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:32:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:32:01.689902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:01.691412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:01.827894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444a383-defd-44d6-b42d-f4ed44e4df7b service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:01.835583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:01.838625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:01.838625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:01.838625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:01.839720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:01.859073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:01.860858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:01.877676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1224/2454] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:32:05.712550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72759cf-750e-4f9a-a615-1f71373bc7eb None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:05.724623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72759cf-750e-4f9a-a615-1f71373bc7eb None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:05.729664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:05.730398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:05.731141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:05.732094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:05.734877 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:05.808342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:05.809841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:05.822032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1231/2455] 10.222.0.42 () {60 vars in 1396 bytes} [Thu May 26 21:32:05 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:11.869042 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:32:15.668478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:15.681401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3163586-e6f1-4733-a8c4-f804f5419561 service placement] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:15.686828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:15.687495 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:15.687810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:15.688093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:15.689836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:15.747699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:15.749706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:15.759194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1225/2456] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:15.815444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=37 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:32:23.308832 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:23.319410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0e90a-05f9-4cb9-b563-dd35f4482e9a service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:23.328442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98325e-535d-413e-a002-b69fa856387f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:23.328442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98325e-535d-413e-a002-b69fa856387f service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:23.328442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98325e-535d-413e-a002-b69fa856387f service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:23.328442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98325e-535d-413e-a002-b69fa856387f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:23.329196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4d98325e-535d-413e-a002-b69fa856387f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:23.340906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98325e-535d-413e-a002-b69fa856387f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:23.342773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98325e-535d-413e-a002-b69fa856387f service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:23.352038 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1232/2457] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:25.754931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=28 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:32:27.754763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:27.824664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca84cc-71f0-49cd-810e-72bea45ee0a7 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:27.829701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:27.830571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:27.830850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:27.831125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] PATH_INFO: `/v3/users/252eb4f862204dd6aca56a7f828b5573` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:27.832330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] RBAC: Authorizing `identity:delete_user(user_id=252eb4f862204dd6aca56a7f828b5573)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:27.834235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:28.019658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252eb4f862204dd6aca56a7f828b5573'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:28.031510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252eb4f862204dd6aca56a7f828b5573'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:28.108973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252eb4f862204dd6aca56a7f828b5573'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:28.109623 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252eb4f862204dd6aca56a7f828b5573'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:28.110259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252eb4f862204dd6aca56a7f828b5573'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:28.126048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1226/2458] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:32:27 2022] DELETE /identity/v3/users/252eb4f862204dd6aca56a7f828b5573 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:32:28.130594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:28.131097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:28.131181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:28.149675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:32:28.150952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:28.220041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1233/2459] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:32:28.245037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:28.246727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:28.331043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07beaa9-3f38-449a-b12c-fd92fa64f39f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:28.335323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9301a-a806-48da-b61c-03e951925322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:28.336834 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9301a-a806-48da-b61c-03e951925322 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:28.337437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9301a-a806-48da-b61c-03e951925322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:28.337984 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9301a-a806-48da-b61c-03e951925322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:28.349265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9301a-a806-48da-b61c-03e951925322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:28.350597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9301a-a806-48da-b61c-03e951925322 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:28.362525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1227/2460] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:28.786324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:28.788101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:28.807577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde7d78-b279-4381-9e04-395b9524094b None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:28.812309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:28.813181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:28.813459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:28.813734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] PATH_INFO: `/v3/projects/954ae0a9b0c64098ae7137d6a63bdb7c` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:28.815012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=954ae0a9b0c64098ae7137d6a63bdb7c)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:28.817006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:28.954033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '954ae0a9b0c64098ae7137d6a63bdb7c', 'request_id': 'req-5c022b71-5f6d-447b-9bac-0b2a42023de0'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:28.960150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '954ae0a9b0c64098ae7137d6a63bdb7c', 'request_id': 'req-5c022b71-5f6d-447b-9bac-0b2a42023de0'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:28.965661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '954ae0a9b0c64098ae7137d6a63bdb7c', 'request_id': 'req-5c022b71-5f6d-447b-9bac-0b2a42023de0'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:29.048368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] The token cache is being invalidate because project 954ae0a9b0c64098ae7137d6a63bdb7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:32:29.048572 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:29.049257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:32:29.051135 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1234/2461] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:32:28 2022] DELETE /identity/v3/projects/954ae0a9b0c64098ae7137d6a63bdb7c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:32:29.271082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:29.271542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:29.271675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:29.289566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:32:29.290872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:29.313205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:32:29.354900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1228/2462] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:32:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:32:29.367226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:29.368710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:29.442376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c022b71-5f6d-447b-9bac-0b2a42023de0 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:29.447686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:29.448385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:29.448665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:29.448946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:29.458473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:29.462327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:29.465684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:32:29.470203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1235/2463] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:32:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:32:31.159653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:31.161586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:31.263463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e03120-1b9a-45f6-910a-de8f2ef0162b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:31.270193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:31.270863 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:31.271145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:31.271419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:31.273570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:31.294815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:31.296674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:31.309499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1229/2464] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:38.742734 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:32:38.744498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:38.821612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a4593-341d-45a0-ad77-2c65ca45d8ca service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:32:38.825252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668ee9c4-0d12-4d0f-8962-34eaf0503d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:32:38.825904 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ee9c4-0d12-4d0f-8962-34eaf0503d89 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:32:38.826184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ee9c4-0d12-4d0f-8962-34eaf0503d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:32:38.826459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668ee9c4-0d12-4d0f-8962-34eaf0503d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:32:38.828298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-668ee9c4-0d12-4d0f-8962-34eaf0503d89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:32:38.838292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ee9c4-0d12-4d0f-8962-34eaf0503d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:32:38.839733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668ee9c4-0d12-4d0f-8962-34eaf0503d89 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:32:38.848947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1236/2465] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:32:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:32:41.304621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=49 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:32:48.844582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=26 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:33:04.763934 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:04.775181 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62833092-4f92-4a88-b09b-e326fdcec545 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:04.780733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:04.781379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:04.781668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:04.781950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:04.788748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:04.796497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:04.798111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:04.810080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1230/2466] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:33:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:33:14.062442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:14.062931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:14.063022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:14.080282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:33:14.082018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:14.089498 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1237/2467] 10.222.0.42 () {58 vars in 967 bytes} [Thu May 26 21:33:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:33:14.111571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:14.122920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703ddf78-5ef9-4fd3-81f5-0774c00f7d75 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:14.128144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:14.128810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:14.129074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:14.129351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:14.137787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:14.139485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:14.148554 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1231/2468] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:33:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:33:24.075358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:33:24.144910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=12 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:33:25.719401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:25.720835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:25.732825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85812f9-b5f1-44a4-8b8c-462cf75634dc None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:25.736466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:25.737113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:25.737394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:25.737669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:25.740111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:25.803345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:25.804825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:25.814595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1238/2469] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:33:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:33:34.674375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:34.686877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed08b20-af1b-4b44-8470-501654030ce5 service swift] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:34.692304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:34.692946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:34.693225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:34.693499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:34.695325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:34.755200 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:34.756823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:34.765898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1232/2470] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:33:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:33:35.810180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=30 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:33:38.995089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:38.996503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:39.054045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904090d-94f1-4b1f-ae51-e7a7e61082bf service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:39.057935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:39.058794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:39.059278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:39.059773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:39.061771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:39.119940 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:39.121456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:39.131218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1239/2471] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:33:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:33:44.761690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=28 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:33:49.126521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=48 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:33:53.749308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:53.759090 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18835c36-5b3d-456d-85ad-fdc178507959 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:53.762616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:53.763260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:53.763542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:53.763852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:53.765674 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:53.831974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:53.833347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:53.844144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1233/2472] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:33:54.407118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:54.408590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:54.478605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95164f2-49e8-42dc-8bca-9e88da8faee1 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:54.483918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:54.484596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:54.484879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:54.485153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:54.486920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:54.498917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:54.500588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:54.510947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1240/2473] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:33:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:33:54.525494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:54.526896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:33:54.540605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ccae8-d48f-4e83-bde8-16171c7b228f service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:54.549918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:54.549918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:54.549918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:54.549918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] PATH_INFO: `/v3/users/6a658b1951a9426d94341940d87b5f95` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:54.550307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a658b1951a9426d94341940d87b5f95)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:54.550307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:54.732624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a658b1951a9426d94341940d87b5f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:54.744404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a658b1951a9426d94341940d87b5f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:54.827010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a658b1951a9426d94341940d87b5f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:54.827632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a658b1951a9426d94341940d87b5f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:54.828176 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9a0579e9-3431-4664-b3e9-0f96dafbf28b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a658b1951a9426d94341940d87b5f95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:54.842344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1234/2474] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:33:54 2022] DELETE /identity/v3/users/6a658b1951a9426d94341940d87b5f95 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:33:55.085543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:33:55.096901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5237cd-e2d5-491f-a6ef-821a09051adc service placement] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:33:55.160611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:33:55.162039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:33:55.162510 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:33:55.162996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] PATH_INFO: `/v3/projects/0a97f689104e4884b6e30b0c8606b513` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:33:55.165062 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a97f689104e4884b6e30b0c8606b513)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:33:55.168454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:33:55.308298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a97f689104e4884b6e30b0c8606b513', 'request_id': 'req-210aefb4-61d0-4477-8f46-c154f79616a3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:55.314372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a97f689104e4884b6e30b0c8606b513', 'request_id': 'req-210aefb4-61d0-4477-8f46-c154f79616a3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:55.320217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a97f689104e4884b6e30b0c8606b513', 'request_id': 'req-210aefb4-61d0-4477-8f46-c154f79616a3'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:55.420825 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] The token cache is being invalidate because project 0a97f689104e4884b6e30b0c8606b513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:33:55.424123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:55.424123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:33:55.424123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1241/2475] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:33:55 2022] DELETE /identity/v3/projects/0a97f689104e4884b6e30b0c8606b513 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:34:04.838766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=59,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:34:05.417525 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:34:05.786384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4ba0e-5af8-4a8e-92d9-eb757291d379 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:34:05.786666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4ba0e-5af8-4a8e-92d9-eb757291d379 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:34:05.786941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4ba0e-5af8-4a8e-92d9-eb757291d379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:34:05.806336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-98e4ba0e-5af8-4a8e-92d9-eb757291d379 None None] MFA Rules not processed for user `ee4307a7bdf84fe18efd6ba3dbe50c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:34:05.807918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-98e4ba0e-5af8-4a8e-92d9-eb757291d379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:34:05.861044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1235/2476] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:34:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:34:15.854531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=25 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:34:45.611974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:34:45.614748 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:34:45.733573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210aefb4-61d0-4477-8f46-c154f79616a3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:34:45.746371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe16443-1b21-4c5c-bb4a-668c507ab4e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:34:45.747080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe16443-1b21-4c5c-bb4a-668c507ab4e3 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:34:45.751452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe16443-1b21-4c5c-bb4a-668c507ab4e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:34:45.751452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe16443-1b21-4c5c-bb4a-668c507ab4e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:34:45.751452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-bfe16443-1b21-4c5c-bb4a-668c507ab4e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:34:45.812298 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe16443-1b21-4c5c-bb4a-668c507ab4e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:34:45.820193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe16443-1b21-4c5c-bb4a-668c507ab4e3 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:34:45.831463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1242/2477] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:34:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:34:47.845927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece40ed9-f2e2-4f49-8688-00b0f8afd361 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:34:47.846494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece40ed9-f2e2-4f49-8688-00b0f8afd361 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:34:47.846594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece40ed9-f2e2-4f49-8688-00b0f8afd361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:34:47.864172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ece40ed9-f2e2-4f49-8688-00b0f8afd361 None None] MFA Rules not processed for user `9b94748ea78342629281f2360a54bee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:34:47.865389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ece40ed9-f2e2-4f49-8688-00b0f8afd361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:34:47.919762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1236/2478] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:34:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:34:51.069500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1921af-feff-4029-ad16-99e5d3e12f34 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:34:51.070158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1921af-feff-4029-ad16-99e5d3e12f34 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:34:51.070158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1921af-feff-4029-ad16-99e5d3e12f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:34:51.099156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-fa1921af-feff-4029-ad16-99e5d3e12f34 None None] MFA Rules not processed for user `88575c0bd9db4fa8bc9bd511c0ac37dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:34:51.100442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fa1921af-feff-4029-ad16-99e5d3e12f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:34:51.126962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1243/2479] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:34:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:34:52.963928 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:34:52.964324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:34:52.964324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:34:52.986359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] MFA Rules not processed for user `6214705ae31e459891f7a215f45a4cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:34:52.987585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:34:53.054471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1237/2480] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:34:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:34:58.561851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1921af-feff-4029-ad16-99e5d3e12f34 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:34:58.578376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1921af-feff-4029-ad16-99e5d3e12f34 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:34:58.605843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:34:58.609950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:34:58.610164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:34:58.610659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:34:58.614073 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:34:58.652240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:34:58.656048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:34:58.671253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1244/2481] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:34:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:01.971416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:01.973073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:02.052425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dc795-b593-4750-8899-a27e1d6884d0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:02.058288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb26b9e-72c4-4ac3-b7e0-dd6efbb1d5cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:02.058996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26b9e-72c4-4ac3-b7e0-dd6efbb1d5cc service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:02.059281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26b9e-72c4-4ac3-b7e0-dd6efbb1d5cc service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:02.059557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb26b9e-72c4-4ac3-b7e0-dd6efbb1d5cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:02.061507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0eb26b9e-72c4-4ac3-b7e0-dd6efbb1d5cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:02.074266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb26b9e-72c4-4ac3-b7e0-dd6efbb1d5cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:02.076449 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb26b9e-72c4-4ac3-b7e0-dd6efbb1d5cc service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:02.089508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1238/2482] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:08.665161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=65 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:12.084979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=82 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:26.808725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:26.810290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:26.826095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da89c9-7a54-4413-87c2-bf8b0beebb45 service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:26.832249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:26.832921 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:26.833211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:26.833493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:26.836078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:26.902468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:26.904254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:26.915086 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1245/2483] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:36.910107 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=28 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:38.480836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:38.482086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:38.482086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:38.500434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:38.501650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:38.560402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1239/2484] 10.222.0.42 () {58 vars in 967 bytes} [Thu May 26 21:35:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:38.692034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:38.693475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:38.764336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b198b5d-8f31-4d34-9119-66bb86479aa7 service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:38.770097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:38.770747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:38.771030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:38.771309 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:38.779831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:38.782082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:38.790236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1246/2485] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:35:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:35:39.965840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:39.967294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:40.025322 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f08cd9-fd30-486e-85b7-230fba70df96 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:40.028937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:40.029581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:40.029865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:40.030690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:40.032592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:40.097103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:40.098545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:40.109019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1240/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:40.320715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:40.322192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:40.334730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f560b5-2077-45d9-9896-20ea9875b142 service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:40.338735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:40.339379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:40.340008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:40.340287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:40.342064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:40.354852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:40.356332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:40.366332 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1247/2487] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:44.050756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:44.052293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:44.081237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651c661-dcd0-44f9-9223-1a7518eab8af service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:44.087926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:44.089065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:44.089528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:44.090007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:44.093850 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:44.136494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:44.143593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:44.157471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1241/2488] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:44.533828 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:44.535319 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:44.548919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b5a49-0494-4fb5-8183-73c797f6f72d service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:44.554078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:44.556501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:44.556501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:44.556501 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] PATH_INFO: `/v3/users/ee4307a7bdf84fe18efd6ba3dbe50c23` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:44.557453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee4307a7bdf84fe18efd6ba3dbe50c23)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:44.559573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:44.700346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee4307a7bdf84fe18efd6ba3dbe50c23'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:44.714812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee4307a7bdf84fe18efd6ba3dbe50c23'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:44.807636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee4307a7bdf84fe18efd6ba3dbe50c23'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:44.808300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee4307a7bdf84fe18efd6ba3dbe50c23'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:44.808915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c407147f-fd1c-4950-bcf0-96f6548e72e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee4307a7bdf84fe18efd6ba3dbe50c23'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:44.822346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1248/2489] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:44 2022] DELETE /identity/v3/users/ee4307a7bdf84fe18efd6ba3dbe50c23 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:45.067549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:45.099849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbecd2-ead3-4004-9289-9e067f3f67a8 service placement] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:45.144201 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:45.163986 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:45.164859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.165144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.165424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] PATH_INFO: `/v3/projects/318999cb1f874176b2523de9686febc1` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.166661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318999cb1f874176b2523de9686febc1)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:45.168633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:45.275281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318999cb1f874176b2523de9686febc1', 'request_id': 'req-a3088d9b-0691-4a94-836b-71c39e954ca5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:45.281530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318999cb1f874176b2523de9686febc1', 'request_id': 'req-a3088d9b-0691-4a94-836b-71c39e954ca5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:45.287571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318999cb1f874176b2523de9686febc1', 'request_id': 'req-a3088d9b-0691-4a94-836b-71c39e954ca5'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:45.361869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] The token cache is being invalidate because project 318999cb1f874176b2523de9686febc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:45.362276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:45.362963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3088d9b-0691-4a94-836b-71c39e954ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:45.364737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1242/2490] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:45 2022] DELETE /identity/v3/projects/318999cb1f874176b2523de9686febc1 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:45.369673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa651d-0b81-4753-b75f-16d14b088c60 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.370088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa651d-0b81-4753-b75f-16d14b088c60 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.370441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa651d-0b81-4753-b75f-16d14b088c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.390685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5bfa651d-0b81-4753-b75f-16d14b088c60 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:45.392041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5bfa651d-0b81-4753-b75f-16d14b088c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:45.437710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:45.453163 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1249/2491] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:45.468790 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbbdb8-d44e-495d-962e-e1d373639c7b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.469213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbbdb8-d44e-495d-962e-e1d373639c7b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.469404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afbbdb8-d44e-495d-962e-e1d373639c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.487932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-0afbbdb8-d44e-495d-962e-e1d373639c7b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:45.491885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0afbbdb8-d44e-495d-962e-e1d373639c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:45.497725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1243/2492] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:45.501969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa651d-0b81-4753-b75f-16d14b088c60 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:45.515203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa651d-0b81-4753-b75f-16d14b088c60 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:45.522749 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:45.523611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.524113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.524492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.525504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:35:45.525875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:45.527631 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:45.545914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1250/2493] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:35:45.549657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbbdb8-d44e-495d-962e-e1d373639c7b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:45.559148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afbbdb8-d44e-495d-962e-e1d373639c7b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:45.565023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:45.565752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.565979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.566299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.567586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:45.569533 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:45.690351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1244/2494] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:35:45 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:45.695258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:45.707286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e16a4-ee2f-48d4-aee3-c6f51f6907d1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:45.768431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:45.769016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.769301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.769577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.770306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:45.772261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:45.860492 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1251/2495] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:35:45 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:45.864689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:45.875679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453ffc-dc05-46a1-afcd-8e60b4fa9009 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:45.880460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:45.881362 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.881648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.881925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.882647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:45.884333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:45.895468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1245/2496] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:45.899730 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:45.912841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe63a83-32c9-42aa-af65-a55c6ad4bb4a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:45.918595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d95683-9593-4ead-9972-2714e9c9e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:45.919725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d95683-9593-4ead-9972-2714e9c9e59a admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.920203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d95683-9593-4ead-9972-2714e9c9e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.921321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d95683-9593-4ead-9972-2714e9c9e59a admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.922463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d95683-9593-4ead-9972-2714e9c9e59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:45.924450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d95683-9593-4ead-9972-2714e9c9e59a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:45.935942 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1252/2497] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:35:45.940150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:45.952864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46292062-d480-4d42-a264-d66e0be887c4 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:45.958075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:45.959125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:45.959412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:45.959690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] PATH_INFO: `/v3/projects/56afe02ca308499fb9665afab9406476/users/d27461356c4d4356a9d964c9d7cf252a/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:45.977821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=56afe02ca308499fb9665afab9406476, user_id=d27461356c4d4356a9d964c9d7cf252a, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:45.980721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:46.044248 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1246/2498] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:45 2022] PUT /identity/v3/projects/56afe02ca308499fb9665afab9406476/users/d27461356c4d4356a9d964c9d7cf252a/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:35:46.057052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd022-0f57-4090-940e-947d833f9b97 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:46.057418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd022-0f57-4090-940e-947d833f9b97 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:46.057623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bd022-0f57-4090-940e-947d833f9b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:46.088180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1b3bd022-0f57-4090-940e-947d833f9b97 None None] MFA Rules not processed for user `d27461356c4d4356a9d964c9d7cf252a`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:46.089686 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1b3bd022-0f57-4090-940e-947d833f9b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:46.168414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1253/2499] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:46.176235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:46.178310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:46.242643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:46.260802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ced3b1-0246-437c-b90a-da714e993fcc admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:46.274564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c28d8-a619-4b3a-bb9f-0dd6d5dc840f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:46.275259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c28d8-a619-4b3a-bb9f-0dd6d5dc840f service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:46.275617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c28d8-a619-4b3a-bb9f-0dd6d5dc840f service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:46.275992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c28d8-a619-4b3a-bb9f-0dd6d5dc840f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:46.286104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c28d8-a619-4b3a-bb9f-0dd6d5dc840f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:46.289441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c28d8-a619-4b3a-bb9f-0dd6d5dc840f service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:46.298787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1247/2500] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:46.565121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd022-0f57-4090-940e-947d833f9b97 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:46.616622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:46.635887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bd022-0f57-4090-940e-947d833f9b97 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:46.642069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:46.643210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:46.643564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:46.643945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] PATH_INFO: `/v3/users/d27461356c4d4356a9d964c9d7cf252a` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:46.645279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27461356c4d4356a9d964c9d7cf252a)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:46.647416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:46.883564 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27461356c4d4356a9d964c9d7cf252a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:46.896019 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27461356c4d4356a9d964c9d7cf252a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:46.979042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27461356c4d4356a9d964c9d7cf252a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:46.979602 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27461356c4d4356a9d964c9d7cf252a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:46.980245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27461356c4d4356a9d964c9d7cf252a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:46.993735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1254/2501] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:46 2022] DELETE /identity/v3/users/d27461356c4d4356a9d964c9d7cf252a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:46.998306 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:46.998666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:46.999503 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:47.017563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:47.018784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:47.089283 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1248/2502] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:47.099909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:47.101385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:47.165765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c07b6-d1c8-4a3b-8082-a978d0e02a4f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:47.169476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79863c-b297-4d42-bdaf-b86027c56a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:47.170138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79863c-b297-4d42-bdaf-b86027c56a08 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:47.170546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79863c-b297-4d42-bdaf-b86027c56a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:47.171383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79863c-b297-4d42-bdaf-b86027c56a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:47.180900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79863c-b297-4d42-bdaf-b86027c56a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:47.182359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79863c-b297-4d42-bdaf-b86027c56a08 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:47.192544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1255/2503] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:47.800473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:47.802843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:47.817286 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506e1a51-7286-478f-bfc3-403d2b23aa4e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:47.822792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:47.827304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:47.827304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:47.827304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] PATH_INFO: `/v3/projects/56afe02ca308499fb9665afab9406476` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:47.827304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56afe02ca308499fb9665afab9406476)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:47.828514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:47.991401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56afe02ca308499fb9665afab9406476', 'request_id': 'req-4901fda6-e1dd-4a63-b862-89f8f0152615'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:47.998487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56afe02ca308499fb9665afab9406476', 'request_id': 'req-4901fda6-e1dd-4a63-b862-89f8f0152615'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.005968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56afe02ca308499fb9665afab9406476', 'request_id': 'req-4901fda6-e1dd-4a63-b862-89f8f0152615'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.082227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] The token cache is being invalidate because project 56afe02ca308499fb9665afab9406476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:48.082802 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.083717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-4901fda6-e1dd-4a63-b862-89f8f0152615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.085665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1249/2504] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:47 2022] DELETE /identity/v3/projects/56afe02ca308499fb9665afab9406476 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:48.095480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf63d2-f8c3-479b-a757-6b1d3f70e979 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.095988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf63d2-f8c3-479b-a757-6b1d3f70e979 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.096181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf63d2-f8c3-479b-a757-6b1d3f70e979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.115588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-fcdf63d2-f8c3-479b-a757-6b1d3f70e979 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:48.117085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fcdf63d2-f8c3-479b-a757-6b1d3f70e979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:48.180126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1256/2505] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:48.189556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.189848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.190122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.209114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:48.210382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:48.218494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1250/2506] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:48.222680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf63d2-f8c3-479b-a757-6b1d3f70e979 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.234206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf63d2-f8c3-479b-a757-6b1d3f70e979 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.240322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.240871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.241161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.241577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.241850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.242392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:48.242719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:35:48.243099 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.245006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.246925 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:48.258622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fe354-eab9-4887-967d-941a9d6b3287 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.259361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1257/2507] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:35:48.263579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.272995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.272995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.272995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.272995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] PATH_INFO: `/v3/users/b9537b7add3b4b5da501903db49a2a11` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.273484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9537b7add3b4b5da501903db49a2a11)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.273484 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.274116 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b606c03-6484-4ae9-b907-1a769b6bb64e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.278805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.279486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.279894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.280281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.281051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.282989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.409239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1258/2508] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:35:48 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:35:48.413417 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.473111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9537b7add3b4b5da501903db49a2a11'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.475120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef0d70-f79e-4442-b704-63cd7700d434 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.479751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.480552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.481236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.481898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:48.482331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.483620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.485654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.492648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9537b7add3b4b5da501903db49a2a11'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.555017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9537b7add3b4b5da501903db49a2a11'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.556045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9537b7add3b4b5da501903db49a2a11'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.556950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9537b7add3b4b5da501903db49a2a11'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:48.563827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1259/2509] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:35:48 2022] POST /identity/v3/users => generated 294 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:35:48.567871 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.569514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:48.575547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1251/2510] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:48 2022] DELETE /identity/v3/users/b9537b7add3b4b5da501903db49a2a11 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:48.583936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d910bc-5a2d-47da-9c94-7afe87c2b564 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.643196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.643909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.644200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.644505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.645212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.646826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.657413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1260/2511] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:35:48.661392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.672288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0692e3-d99a-40b8-86e4-a5e7dc6acb4c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.677531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e42107-689e-4a10-a644-53b73aee82d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.678016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e42107-689e-4a10-a644-53b73aee82d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.678299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e42107-689e-4a10-a644-53b73aee82d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.678576 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e42107-689e-4a10-a644-53b73aee82d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.679301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e42107-689e-4a10-a644-53b73aee82d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.680937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e42107-689e-4a10-a644-53b73aee82d3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.691464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1252/2512] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:48.695632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.706642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647052b-61ca-417c-bcfd-c45e0d1a48a1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.711929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.712862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.713153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.713409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] PATH_INFO: `/v3/projects/6a730091b4ad4c02afac3e184b73f760/users/d27fd365d8e5467889fab16a2a0e1b60/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.730980 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a730091b4ad4c02afac3e184b73f760, user_id=d27fd365d8e5467889fab16a2a0e1b60, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.734194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.836162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1261/2513] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:48 2022] PUT /identity/v3/projects/6a730091b4ad4c02afac3e184b73f760/users/d27fd365d8e5467889fab16a2a0e1b60/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:48.847335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6d3bf-2e7d-46a6-8ebe-fb6ca603138f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.847814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6d3bf-2e7d-46a6-8ebe-fb6ca603138f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.848025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6d3bf-2e7d-46a6-8ebe-fb6ca603138f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.849960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.855351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:48.887928 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-61b6d3bf-2e7d-46a6-8ebe-fb6ca603138f None None] MFA Rules not processed for user `d27fd365d8e5467889fab16a2a0e1b60`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:48.889210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-61b6d3bf-2e7d-46a6-8ebe-fb6ca603138f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:48.892305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:48.922312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e115f6b-a110-4597-84ad-c3a8f9795dbf admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.927596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.928607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.928892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.929165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] PATH_INFO: `/v3/projects/771d85ed5c3b4a6bbc9945d88cfcbab8` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.930372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=771d85ed5c3b4a6bbc9945d88cfcbab8)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.932605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:48.959556 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1253/2514] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:48.964104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6d3bf-2e7d-46a6-8ebe-fb6ca603138f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:48.975110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6d3bf-2e7d-46a6-8ebe-fb6ca603138f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:48.980128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:48.980779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:48.981115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:48.981341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:48.982061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:48.984624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.063841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '771d85ed5c3b4a6bbc9945d88cfcbab8', 'request_id': 'req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.072794 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '771d85ed5c3b4a6bbc9945d88cfcbab8', 'request_id': 'req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.083239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1254/2515] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:35:48 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:35:49.083470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '771d85ed5c3b4a6bbc9945d88cfcbab8', 'request_id': 'req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.084650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.144690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] The token cache is being invalidate because project 771d85ed5c3b4a6bbc9945d88cfcbab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:49.145244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.145746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.146400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326322a-cf0a-483f-990b-5557e7cd6236 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.147323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1262/2516] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:48 2022] DELETE /identity/v3/projects/771d85ed5c3b4a6bbc9945d88cfcbab8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:35:49.151210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.152752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.152913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:49.153431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.153868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.154174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.155571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.157617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.228983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd46a447-94ec-4b0f-9e5a-f07541b8d9d4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.233770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.234578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.234855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.235126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] PATH_INFO: `/v3/users/0d804c83a57244b595447f8400d2fa28` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.236337 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d804c83a57244b595447f8400d2fa28)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.236539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1255/2517] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:35:49 2022] POST /identity/v3/users => generated 299 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:49.238695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.240366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.242065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:49.254325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c1322-2162-4ec6-89f1-832b30b5b788 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.259126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.259904 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.260253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.260534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.261266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.262867 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.273701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1256/2518] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:35:49.277915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.288308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7eaa7-aa03-4218-a64b-8f13537488cb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.292870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.293784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.294064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.294342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] PATH_INFO: `/v3/projects/c26544af5894409aa3770e82c47d76d4/users/7fa67af4db444befb4585285a2916e95/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.297673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:49.309276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26544af5894409aa3770e82c47d76d4, user_id=7fa67af4db444befb4585285a2916e95, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.312002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.358722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d804c83a57244b595447f8400d2fa28'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.370283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d804c83a57244b595447f8400d2fa28'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.372176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:49.380844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1257/2519] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:49 2022] PUT /identity/v3/projects/c26544af5894409aa3770e82c47d76d4/users/7fa67af4db444befb4585285a2916e95/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:35:49.384726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.446685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cea09d-ee0f-464c-a06e-1e3d709d09b7 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.451452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.452382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d804c83a57244b595447f8400d2fa28'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.452708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.452998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.453257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d804c83a57244b595447f8400d2fa28'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.453382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.453951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-007f6376-469b-46ed-8cd4-6433fd068920 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d804c83a57244b595447f8400d2fa28'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:49.454077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.455528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.468523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1258/2520] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:49 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:35:49.469508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.471175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:49.471817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1263/2521] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:49 2022] DELETE /identity/v3/users/0d804c83a57244b595447f8400d2fa28 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:35:49.484499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada15db2-2327-4924-98a8-9048859e405c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.545511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.546416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.546695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.547280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] PATH_INFO: `/v3/projects/c26544af5894409aa3770e82c47d76d4/users/7fa67af4db444befb4585285a2916e95/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.549339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26544af5894409aa3770e82c47d76d4, user_id=7fa67af4db444befb4585285a2916e95, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.552028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.564347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1259/2522] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:49 2022] PUT /identity/v3/projects/c26544af5894409aa3770e82c47d76d4/users/7fa67af4db444befb4585285a2916e95/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:35:49.574830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90330938-bef9-44d2-8280-536c627d706f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.575120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90330938-bef9-44d2-8280-536c627d706f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.575396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90330938-bef9-44d2-8280-536c627d706f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.606907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-90330938-bef9-44d2-8280-536c627d706f None None] MFA Rules not processed for user `7fa67af4db444befb4585285a2916e95`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:49.608324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-90330938-bef9-44d2-8280-536c627d706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:49.678045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1264/2523] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:49.691433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.693186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:49.741937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90330938-bef9-44d2-8280-536c627d706f None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.743773 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-90330938-bef9-44d2-8280-536c627d706f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:49.758655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5426161-2d9e-4d18-bbb5-c2b61e7dd01b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.763544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.768397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.768705 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.768890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.780600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.782031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.792311 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1260/2524] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:49.821294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90330938-bef9-44d2-8280-536c627d706f None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.827233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.828263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.828603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.828939 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] PATH_INFO: `/v3/projects/17c68dbe9a4a402685d6e60409d6b3a7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.830267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17c68dbe9a4a402685d6e60409d6b3a7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.833732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.896648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:49.907930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ba6f2-3715-4040-82eb-bfcd17fd34ad service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:49.912596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:49.913451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:49.913728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:49.914013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] PATH_INFO: `/v3/users/d27fd365d8e5467889fab16a2a0e1b60` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:49.915182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27fd365d8e5467889fab16a2a0e1b60)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:49.917097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:49.931050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:50.105335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17c68dbe9a4a402685d6e60409d6b3a7', 'request_id': 'req-ac0e5d63-016a-46c7-a9a4-9363a0687e80'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.112143 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17c68dbe9a4a402685d6e60409d6b3a7', 'request_id': 'req-ac0e5d63-016a-46c7-a9a4-9363a0687e80'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.118246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17c68dbe9a4a402685d6e60409d6b3a7', 'request_id': 'req-ac0e5d63-016a-46c7-a9a4-9363a0687e80'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.237329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:50.242521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] The token cache is being invalidate because project 17c68dbe9a4a402685d6e60409d6b3a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:50.242812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.243485 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ac0e5d63-016a-46c7-a9a4-9363a0687e80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.245041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1265/2525] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:49 2022] DELETE /identity/v3/projects/17c68dbe9a4a402685d6e60409d6b3a7 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:50.252279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27fd365d8e5467889fab16a2a0e1b60'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.266257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27fd365d8e5467889fab16a2a0e1b60'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.350682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27fd365d8e5467889fab16a2a0e1b60'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.351364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27fd365d8e5467889fab16a2a0e1b60'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.352014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27fd365d8e5467889fab16a2a0e1b60'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:50.366990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1261/2526] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:49 2022] DELETE /identity/v3/users/d27fd365d8e5467889fab16a2a0e1b60 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:35:50.371571 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:50.374865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:50.374865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:50.391545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:50.392936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:50.459206 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1266/2527] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:50.471080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:50.472728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:50.531907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060e7b4-b9c8-46bf-a872-004f8f07e133 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:50.535462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:50.536129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:50.536412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:50.536686 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:50.546110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:50.547800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:50.557207 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1262/2528] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:50.941170 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:50.946030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:50.955508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fbb68-c6e8-4a38-a625-0b1230450bb5 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:50.961321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:50.962305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:50.962685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:50.963025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] PATH_INFO: `/v3/projects/6a730091b4ad4c02afac3e184b73f760` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:50.964456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a730091b4ad4c02afac3e184b73f760)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:50.969516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:51.105956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a730091b4ad4c02afac3e184b73f760', 'request_id': 'req-b86184e2-da87-4c57-9599-e871f53199e9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.112763 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a730091b4ad4c02afac3e184b73f760', 'request_id': 'req-b86184e2-da87-4c57-9599-e871f53199e9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.118554 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a730091b4ad4c02afac3e184b73f760', 'request_id': 'req-b86184e2-da87-4c57-9599-e871f53199e9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.213175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] The token cache is being invalidate because project 6a730091b4ad4c02afac3e184b73f760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:51.213390 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.213971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.215643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1267/2529] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:50 2022] DELETE /identity/v3/projects/6a730091b4ad4c02afac3e184b73f760 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:51.219636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:51.229629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7ecfd1-f7f2-4698-ab5e-e9d2f0588b0c service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:51.285609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:51.286477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:51.286757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:51.287028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] PATH_INFO: `/v3/users/7fa67af4db444befb4585285a2916e95` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:51.288257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fa67af4db444befb4585285a2916e95)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:51.290144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:51.469558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:51.478609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fa67af4db444befb4585285a2916e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.491005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fa67af4db444befb4585285a2916e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.567374 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fa67af4db444befb4585285a2916e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.568043 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fa67af4db444befb4585285a2916e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.568640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-ce768343-044e-4b96-acf3-85558a67c63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fa67af4db444befb4585285a2916e95'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:51.581164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1263/2530] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:51 2022] DELETE /identity/v3/users/7fa67af4db444befb4585285a2916e95 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:51.999534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:51.999534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:52.012902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86184e2-da87-4c57-9599-e871f53199e9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:52.081515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:52.082367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.082647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.082918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] PATH_INFO: `/v3/projects/c26544af5894409aa3770e82c47d76d4` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.084142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26544af5894409aa3770e82c47d76d4)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:52.086079 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:52.110313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:52.220066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26544af5894409aa3770e82c47d76d4', 'request_id': 'req-74e01735-0cc0-4e09-b120-38d37542d9e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:52.226575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26544af5894409aa3770e82c47d76d4', 'request_id': 'req-74e01735-0cc0-4e09-b120-38d37542d9e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:52.232518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26544af5894409aa3770e82c47d76d4', 'request_id': 'req-74e01735-0cc0-4e09-b120-38d37542d9e8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:52.303815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] The token cache is being invalidate because project c26544af5894409aa3770e82c47d76d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:52.304042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:52.304733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-74e01735-0cc0-4e09-b120-38d37542d9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:52.306303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1268/2531] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:51 2022] DELETE /identity/v3/projects/c26544af5894409aa3770e82c47d76d4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:52.311192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f9bb8-94b7-46ae-b0bf-bd6a3a7f4b6d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.311668 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f9bb8-94b7-46ae-b0bf-bd6a3a7f4b6d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.311783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f9bb8-94b7-46ae-b0bf-bd6a3a7f4b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.330629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-147f9bb8-94b7-46ae-b0bf-bd6a3a7f4b6d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:52.331840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-147f9bb8-94b7-46ae-b0bf-bd6a3a7f4b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:52.399508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1264/2532] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:35:52.399654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150456f2-54cb-4660-a4ec-3959b61e50f4 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.399654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150456f2-54cb-4660-a4ec-3959b61e50f4 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.399654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150456f2-54cb-4660-a4ec-3959b61e50f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.417533 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-150456f2-54cb-4660-a4ec-3959b61e50f4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:52.418738 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-150456f2-54cb-4660-a4ec-3959b61e50f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:52.427109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1269/2533] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:52.431384 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f9bb8-94b7-46ae-b0bf-bd6a3a7f4b6d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:52.441622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f9bb8-94b7-46ae-b0bf-bd6a3a7f4b6d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:52.446438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:52.451127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.451127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.451127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.451127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:35:52.451127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:52.451127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:52.462351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1265/2534] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:35:52.466754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150456f2-54cb-4660-a4ec-3959b61e50f4 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:52.476254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150456f2-54cb-4660-a4ec-3959b61e50f4 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:52.481105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:52.481762 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.482047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.482325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.483057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:52.485050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:52.563959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1270/2535] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:35:52 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:52.568188 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:52.578711 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeef1b7-13a6-4241-9288-04c2075fae62 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:52.635364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:52.636070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.636355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.636632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.637328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:52.639253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:52.733605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1266/2536] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:35:52 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:52.738365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:52.748884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd1778-9e42-4be3-8abe-8148abf0a958 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:52.754264 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:52.754924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.755233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.755511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.756274 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:52.757881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:52.768370 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1271/2537] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:52.773421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:52.783401 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee09fd-74e3-42f1-84fa-8d1692d428cd admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:52.788018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7b9ea-fcdc-49b4-a048-18c2f2a51e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:52.788672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b9ea-fcdc-49b4-a048-18c2f2a51e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.788953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b9ea-fcdc-49b4-a048-18c2f2a51e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.789223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7b9ea-fcdc-49b4-a048-18c2f2a51e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.789955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7b9ea-fcdc-49b4-a048-18c2f2a51e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:52.791513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7b9ea-fcdc-49b4-a048-18c2f2a51e93 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:52.802178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1267/2538] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:52.806289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:52.817150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12b1ab-4cd6-434c-a159-a0adab3ab110 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:52.821974 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:52.822849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.823260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.823539 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] PATH_INFO: `/v3/projects/391147520fc04bb3853b72b07b15a180/users/e9b7703cc7914d35bb959581614c343c/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.838399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=391147520fc04bb3853b72b07b15a180, user_id=e9b7703cc7914d35bb959581614c343c, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:52.841263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:52.928603 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1272/2539] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:52 2022] PUT /identity/v3/projects/391147520fc04bb3853b72b07b15a180/users/e9b7703cc7914d35bb959581614c343c/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:52.939486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81195-1dac-4122-8713-a9a1c66b3fc9 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:52.939807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81195-1dac-4122-8713-a9a1c66b3fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:52.940119 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81195-1dac-4122-8713-a9a1c66b3fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:52.966597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d6e81195-1dac-4122-8713-a9a1c66b3fc9 None None] MFA Rules not processed for user `e9b7703cc7914d35bb959581614c343c`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:52.967836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d6e81195-1dac-4122-8713-a9a1c66b3fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:52.972748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:53.030868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1268/2540] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:53.035443 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:53.098122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb91971-a937-44e0-943d-248d7ddfa138 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:53.102765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:53.103425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.103707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.104011 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.104727 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:53.106615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:53.230330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1273/2541] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:35:53 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:53.235069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81195-1dac-4122-8713-a9a1c66b3fc9 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:53.245992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81195-1dac-4122-8713-a9a1c66b3fc9 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:53.300611 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:53.301271 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.301552 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.301824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.302499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:53.304428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:53.378540 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1269/2542] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:35:53 2022] POST /identity/v3/users => generated 300 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:53.381894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:53.392313 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4d180-ae19-4b85-9665-eb87fc0420e5 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:53.396908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:53.397593 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.397884 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.398162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.398910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:53.400588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:53.411692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1274/2543] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:53.415952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:53.426688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8698ee5-31cc-47ea-a1ae-83fd589a1399 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:53.431246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:53.432838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.432838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.432838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] PATH_INFO: `/v3/projects/d6736808e31c43358e72aac473791fa5/users/e985a99d03764d19be747dfa2416e73a/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.447402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6736808e31c43358e72aac473791fa5, user_id=e985a99d03764d19be747dfa2416e73a, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:53.450044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:53.545319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1270/2544] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:53 2022] PUT /identity/v3/projects/d6736808e31c43358e72aac473791fa5/users/e985a99d03764d19be747dfa2416e73a/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:53.549766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:53.560004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541c41-0bb2-4976-8845-de692deba46e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:53.579115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:53.620760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529be027-847a-4101-ae48-4e8c4d058d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:53.621433 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529be027-847a-4101-ae48-4e8c4d058d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.621731 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529be027-847a-4101-ae48-4e8c4d058d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.621993 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529be027-847a-4101-ae48-4e8c4d058d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.622720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529be027-847a-4101-ae48-4e8c4d058d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:53.624434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529be027-847a-4101-ae48-4e8c4d058d90 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:53.634326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1275/2545] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:53.638583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:53.650024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73ec33-9800-4907-b78f-95ae8fcf3e6b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:53.654749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:53.657234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.657234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.657234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] PATH_INFO: `/v3/projects/d6736808e31c43358e72aac473791fa5/users/e985a99d03764d19be747dfa2416e73a/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.658971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6736808e31c43358e72aac473791fa5, user_id=e985a99d03764d19be747dfa2416e73a, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:53.661702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:53.672509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1271/2546] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:53 2022] PUT /identity/v3/projects/d6736808e31c43358e72aac473791fa5/users/e985a99d03764d19be747dfa2416e73a/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:53.683647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fc22c-1ba6-4d59-a750-7dbc07af8273 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.684001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fc22c-1ba6-4d59-a750-7dbc07af8273 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.684267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fc22c-1ba6-4d59-a750-7dbc07af8273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.711889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6c8fc22c-1ba6-4d59-a750-7dbc07af8273 None None] MFA Rules not processed for user `e985a99d03764d19be747dfa2416e73a`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:53.713152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6c8fc22c-1ba6-4d59-a750-7dbc07af8273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:53.781929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1276/2547] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:53.792625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:53.794339 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:53.852480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3025e-46cf-4b14-941a-c3787d444d52 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:53.856136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92918a8b-228c-4364-b71d-5a8f754bd43e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:53.856775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92918a8b-228c-4364-b71d-5a8f754bd43e service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:53.857075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92918a8b-228c-4364-b71d-5a8f754bd43e service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:53.857349 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92918a8b-228c-4364-b71d-5a8f754bd43e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:53.866583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92918a8b-228c-4364-b71d-5a8f754bd43e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:53.869266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92918a8b-228c-4364-b71d-5a8f754bd43e service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:53.878228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1272/2548] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:54.964827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fc22c-1ba6-4d59-a750-7dbc07af8273 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:55.027434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fc22c-1ba6-4d59-a750-7dbc07af8273 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:55.032691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:55.033615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:55.033847 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:55.034123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] PATH_INFO: `/v3/users/e9b7703cc7914d35bb959581614c343c` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:55.035308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9b7703cc7914d35bb959581614c343c)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:55.037265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:55.259457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:55.260992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9b7703cc7914d35bb959581614c343c'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:55.273454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9b7703cc7914d35bb959581614c343c'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:55.365495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9b7703cc7914d35bb959581614c343c'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:55.366048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9b7703cc7914d35bb959581614c343c'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:55.366644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9b7703cc7914d35bb959581614c343c'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:55.379612 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1277/2549] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:54 2022] DELETE /identity/v3/users/e9b7703cc7914d35bb959581614c343c => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:55.384254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:55.384755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:55.384853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:55.391126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:55.403266 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:55.404505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:55.462992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1273/2550] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:55.472523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:55.473950 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:55.531900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa148c0e-7964-431c-b3a5-ce35d3fa3029 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:55.535538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:55.536200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:55.536480 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:55.536752 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:55.546069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:55.547775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:55.557122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1278/2551] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:35:55.997773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:55.998186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:56.010669 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5211fe-f58d-48df-8472-79cecf154d89 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:56.015269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:56.016194 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:56.016465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:56.016733 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] PATH_INFO: `/v3/projects/391147520fc04bb3853b72b07b15a180` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:56.017942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=391147520fc04bb3853b72b07b15a180)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:56.019925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:56.156703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '391147520fc04bb3853b72b07b15a180', 'request_id': 'req-6ba9a949-8e95-4fd5-ae04-355b95f07b33'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.162845 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '391147520fc04bb3853b72b07b15a180', 'request_id': 'req-6ba9a949-8e95-4fd5-ae04-355b95f07b33'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.169827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '391147520fc04bb3853b72b07b15a180', 'request_id': 'req-6ba9a949-8e95-4fd5-ae04-355b95f07b33'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.262996 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] The token cache is being invalidate because project 391147520fc04bb3853b72b07b15a180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:56.263210 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.263858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.266154 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1274/2552] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:55 2022] DELETE /identity/v3/projects/391147520fc04bb3853b72b07b15a180 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:56.270080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:56.279879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7dfefd-e87b-4937-a21e-1f74983f15c3 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:56.335634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:56.336517 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:56.336799 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:56.337077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] PATH_INFO: `/v3/users/e985a99d03764d19be747dfa2416e73a` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:56.338260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e985a99d03764d19be747dfa2416e73a)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:56.340236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:56.509495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e985a99d03764d19be747dfa2416e73a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.526678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e985a99d03764d19be747dfa2416e73a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.529265 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:56.573222 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e985a99d03764d19be747dfa2416e73a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.573839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e985a99d03764d19be747dfa2416e73a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.574465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-44131867-6383-4648-803e-84420c0eea46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e985a99d03764d19be747dfa2416e73a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:56.587248 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1279/2553] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:35:56 2022] DELETE /identity/v3/users/e985a99d03764d19be747dfa2416e73a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:56.973854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:56.975434 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:56.989939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9a949-8e95-4fd5-ae04-355b95f07b33 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:57.047258 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:57.048096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.048379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.048651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] PATH_INFO: `/v3/projects/d6736808e31c43358e72aac473791fa5` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.049848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6736808e31c43358e72aac473791fa5)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:57.051735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:57.151007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:57.166069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6736808e31c43358e72aac473791fa5', 'request_id': 'req-720a3f24-6c48-477f-823a-9d50bae5ff23'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:57.172871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6736808e31c43358e72aac473791fa5', 'request_id': 'req-720a3f24-6c48-477f-823a-9d50bae5ff23'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:57.178348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6736808e31c43358e72aac473791fa5', 'request_id': 'req-720a3f24-6c48-477f-823a-9d50bae5ff23'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:57.257769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] The token cache is being invalidate because project d6736808e31c43358e72aac473791fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:35:57.258114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:57.258769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-720a3f24-6c48-477f-823a-9d50bae5ff23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:35:57.260357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1275/2554] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:35:56 2022] DELETE /identity/v3/projects/d6736808e31c43358e72aac473791fa5 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:57.265882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd266-150b-414e-ae35-0f41d0db57f0 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.268290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd266-150b-414e-ae35-0f41d0db57f0 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.268290 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517cd266-150b-414e-ae35-0f41d0db57f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.284898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-517cd266-150b-414e-ae35-0f41d0db57f0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:57.286138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-517cd266-150b-414e-ae35-0f41d0db57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:57.345760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1280/2555] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:57.353673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5aafc5-45d5-448d-a3df-75aaa86257f0 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.353971 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5aafc5-45d5-448d-a3df-75aaa86257f0 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.354256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5aafc5-45d5-448d-a3df-75aaa86257f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.374483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6c5aafc5-45d5-448d-a3df-75aaa86257f0 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:57.375719 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6c5aafc5-45d5-448d-a3df-75aaa86257f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:57.383655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1276/2556] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:57.388047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cd266-150b-414e-ae35-0f41d0db57f0 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:57.398422 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517cd266-150b-414e-ae35-0f41d0db57f0 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:57.403002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:57.403655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.403973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.404252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.404978 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:35:57.405287 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:57.407523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:57.419004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1281/2557] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:35:57.423693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5aafc5-45d5-448d-a3df-75aaa86257f0 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:57.432969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5aafc5-45d5-448d-a3df-75aaa86257f0 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:57.437679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:57.438331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.438618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.438895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.439579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:57.441483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:57.529424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1277/2558] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:35:57 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:35:57.534626 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:57.545205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31087f10-64aa-468f-966c-23040a8d15bf admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:57.613244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:57.614072 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.614356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.617518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.617518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:57.617518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:57.688187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1282/2559] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:35:57 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:35:57.692424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:57.706667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea35f0-dd72-4f25-b7b7-1a9aa93db1ee admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:57.711256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:57.711962 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.712246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.712521 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.713240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:57.714817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:57.725835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1278/2560] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:57.729949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:57.740870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25414d-612f-4ff6-a616-99352cd40f19 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:57.745543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb8c45d-c5b6-42b1-bb86-f66a0286b7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:57.746193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c45d-c5b6-42b1-bb86-f66a0286b7ec admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.746475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c45d-c5b6-42b1-bb86-f66a0286b7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.746750 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb8c45d-c5b6-42b1-bb86-f66a0286b7ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.747470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8c45d-c5b6-42b1-bb86-f66a0286b7ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:57.749097 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb8c45d-c5b6-42b1-bb86-f66a0286b7ec admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:57.759257 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1283/2561] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:35:57.763469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:57.773634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52b4bb-4d09-4d35-93c8-970d0a090607 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:57.778341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:57.779213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.779499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.779804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] PATH_INFO: `/v3/projects/8ef34c4893d44e09b6f9a49e3f4a0f84/users/b3f709a27c5f47c1bbc7afad83255734/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.794706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ef34c4893d44e09b6f9a49e3f4a0f84, user_id=b3f709a27c5f47c1bbc7afad83255734, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:57.798000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:57.865901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1279/2562] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:35:57 2022] PUT /identity/v3/projects/8ef34c4893d44e09b6f9a49e3f4a0f84/users/b3f709a27c5f47c1bbc7afad83255734/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:35:57.876167 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b017710-155a-41e2-a03b-b5c266234db2 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:57.876495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b017710-155a-41e2-a03b-b5c266234db2 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:57.876790 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b017710-155a-41e2-a03b-b5c266234db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:57.904358 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7b017710-155a-41e2-a03b-b5c266234db2 None None] MFA Rules not processed for user `b3f709a27c5f47c1bbc7afad83255734`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:35:57.905757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7b017710-155a-41e2-a03b-b5c266234db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:57.946532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:35:57.972391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1284/2563] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:35:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:35:57.983264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:35:57.985070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:35:58.042180 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806f245c-ce4f-4aee-81e1-b16e0db2efb8 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:35:58.046400 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54a505-e51b-4ca6-a403-92316918a3a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:35:58.047340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a505-e51b-4ca6-a403-92316918a3a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:35:58.047616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a505-e51b-4ca6-a403-92316918a3a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:35:58.047926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54a505-e51b-4ca6-a403-92316918a3a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:35:58.057135 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54a505-e51b-4ca6-a403-92316918a3a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:35:58.058589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c54a505-e51b-4ca6-a403-92316918a3a8 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:35:58.067633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1280/2564] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:35:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:08.063585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=67,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:08.075569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=9 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:08.784583 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b017710-155a-41e2-a03b-b5c266234db2 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:08.845188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b017710-155a-41e2-a03b-b5c266234db2 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:08.850130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:08.850984 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:08.851262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:08.851534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] PATH_INFO: `/v3/users/b3f709a27c5f47c1bbc7afad83255734` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:08.852740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f709a27c5f47c1bbc7afad83255734)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:08.854651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:09.076697 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f709a27c5f47c1bbc7afad83255734'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.089127 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f709a27c5f47c1bbc7afad83255734'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.154082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f709a27c5f47c1bbc7afad83255734'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.154869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f709a27c5f47c1bbc7afad83255734'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.155586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f709a27c5f47c1bbc7afad83255734'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.169105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1285/2565] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:36:08 2022] DELETE /identity/v3/users/b3f709a27c5f47c1bbc7afad83255734 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:09.173948 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:09.174435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:09.174555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:09.200932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:09.202178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:09.259438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1281/2566] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:09.269104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:09.271041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:09.328425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924902f-c5a6-4ae8-bb89-43dcf5287541 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:09.332552 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137e025-1d22-4298-b299-9af835f540e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:09.333102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e025-1d22-4298-b299-9af835f540e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:09.333384 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e025-1d22-4298-b299-9af835f540e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:09.333657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e025-1d22-4298-b299-9af835f540e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:09.343398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137e025-1d22-4298-b299-9af835f540e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:09.344875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137e025-1d22-4298-b299-9af835f540e0 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:09.355584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1286/2567] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:09.757847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:09.759321 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:09.774027 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5bd9b8-3d19-4a73-9515-538adca9820e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:09.779031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:09.780002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:09.780252 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:09.783337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] PATH_INFO: `/v3/projects/8ef34c4893d44e09b6f9a49e3f4a0f84` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:09.783337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ef34c4893d44e09b6f9a49e3f4a0f84)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:09.783596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:09.883599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ef34c4893d44e09b6f9a49e3f4a0f84', 'request_id': 'req-b5b08c8f-0110-42e8-adee-1e6dfce96895'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.889740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ef34c4893d44e09b6f9a49e3f4a0f84', 'request_id': 'req-b5b08c8f-0110-42e8-adee-1e6dfce96895'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.895223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ef34c4893d44e09b6f9a49e3f4a0f84', 'request_id': 'req-b5b08c8f-0110-42e8-adee-1e6dfce96895'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.967562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] The token cache is being invalidate because project 8ef34c4893d44e09b6f9a49e3f4a0f84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:36:09.967911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.968542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-b5b08c8f-0110-42e8-adee-1e6dfce96895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:09.970098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1282/2568] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:36:09 2022] DELETE /identity/v3/projects/8ef34c4893d44e09b6f9a49e3f4a0f84 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:09.975538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d8227-d643-49c9-83bc-3371594a23ad None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:09.975866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d8227-d643-49c9-83bc-3371594a23ad None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:09.976263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0d8227-d643-49c9-83bc-3371594a23ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:09.994667 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5e0d8227-d643-49c9-83bc-3371594a23ad None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:09.995919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5e0d8227-d643-49c9-83bc-3371594a23ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:10.014353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:10.055670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1287/2569] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:10.064221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a18f9-dded-4806-952a-c929cbbf1ca7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.064513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a18f9-dded-4806-952a-c929cbbf1ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.064795 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a18f9-dded-4806-952a-c929cbbf1ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.085170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-763a18f9-dded-4806-952a-c929cbbf1ca7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:10.091770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-763a18f9-dded-4806-952a-c929cbbf1ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:10.094366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1283/2570] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:10.099853 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d8227-d643-49c9-83bc-3371594a23ad None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.109780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0d8227-d643-49c9-83bc-3371594a23ad None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.114446 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.115113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.115397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.115677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.116438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:36:10.120372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.120372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:10.130342 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1288/2571] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:36:10.134687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a18f9-dded-4806-952a-c929cbbf1ca7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.144078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a18f9-dded-4806-952a-c929cbbf1ca7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.149205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.149852 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.150132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.150403 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.151093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.153000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:10.255310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1284/2572] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:36:10 2022] POST /identity/v3/projects => generated 359 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:36:10.259351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.270102 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903886d3-6fbb-4926-850e-dcfef2c3431b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.325986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.326735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.326990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.327231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.327963 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.329907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:10.407356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1289/2573] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:36:10 2022] POST /identity/v3/users => generated 280 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:36:10.411443 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.422648 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9509bb-2361-469c-b992-e7112ca09036 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.427296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.427992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.428308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.428655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.429473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.430999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:10.441928 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1285/2574] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:36:10.445931 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.456096 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49399a0-558d-4c5e-b11f-c1908fa56964 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.461368 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8b145-a6df-43f5-ac8e-fd5020e65f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.462024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8b145-a6df-43f5-ac8e-fd5020e65f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.462307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8b145-a6df-43f5-ac8e-fd5020e65f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.462629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8b145-a6df-43f5-ac8e-fd5020e65f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.463345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8b145-a6df-43f5-ac8e-fd5020e65f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.464988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8b145-a6df-43f5-ac8e-fd5020e65f7f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:10.477122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1290/2575] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:36:10.481302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.492840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6a4a6e-b36d-4297-a7e0-40ee046c8f1b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.496979 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.498041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.498351 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.498626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] PATH_INFO: `/v3/projects/db4b5f80260d4c58a745f3e4934dfa72/users/6a95cb933c0f4250b2b0179c2d99eb5f/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.514706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db4b5f80260d4c58a745f3e4934dfa72, user_id=6a95cb933c0f4250b2b0179c2d99eb5f, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.517578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:10.607998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1286/2576] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:36:10 2022] PUT /identity/v3/projects/db4b5f80260d4c58a745f3e4934dfa72/users/6a95cb933c0f4250b2b0179c2d99eb5f/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:10.619346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace6e10-fab5-4db1-a3ba-18f5d5c05109 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.619765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace6e10-fab5-4db1-a3ba-18f5d5c05109 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.620118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace6e10-fab5-4db1-a3ba-18f5d5c05109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.648407 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5ace6e10-fab5-4db1-a3ba-18f5d5c05109 None None] MFA Rules not processed for user `6a95cb933c0f4250b2b0179c2d99eb5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:10.649619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5ace6e10-fab5-4db1-a3ba-18f5d5c05109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:10.715634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1291/2577] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:10 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:10.747604 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.749428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:10.772023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:10.808149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc7256-94b8-4d25-b14f-d22b023b03c5 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.811855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48df9d46-dca6-4efa-8ae6-0ef28e68f097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.812500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9d46-dca6-4efa-8ae6-0ef28e68f097 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.812780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9d46-dca6-4efa-8ae6-0ef28e68f097 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.813051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48df9d46-dca6-4efa-8ae6-0ef28e68f097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.822013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df9d46-dca6-4efa-8ae6-0ef28e68f097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.823424 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48df9d46-dca6-4efa-8ae6-0ef28e68f097 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:10.832081 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1287/2578] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:10 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:10.845851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace6e10-fab5-4db1-a3ba-18f5d5c05109 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:10.865405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:10.907798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ace6e10-fab5-4db1-a3ba-18f5d5c05109 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:10.912525 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:10.913385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:10.913698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:10.913943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] PATH_INFO: `/v3/users/6a95cb933c0f4250b2b0179c2d99eb5f` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:10.915160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a95cb933c0f4250b2b0179c2d99eb5f)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:10.917128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:11.133934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a95cb933c0f4250b2b0179c2d99eb5f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.146843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a95cb933c0f4250b2b0179c2d99eb5f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.207009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a95cb933c0f4250b2b0179c2d99eb5f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.207637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a95cb933c0f4250b2b0179c2d99eb5f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.208282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a95cb933c0f4250b2b0179c2d99eb5f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.221005 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1292/2579] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:36:10 2022] DELETE /identity/v3/users/6a95cb933c0f4250b2b0179c2d99eb5f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:11.226019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:11.226305 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:11.226581 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:11.243960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:11.245163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:11.303195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1288/2580] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:11.312715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:11.314156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:11.371131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c29ddb-68f2-4052-b37a-94364f95a882 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:11.374725 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717883a4-e5f3-40b7-bb1b-de3d2c7abe77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:11.375362 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717883a4-e5f3-40b7-bb1b-de3d2c7abe77 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:11.375645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717883a4-e5f3-40b7-bb1b-de3d2c7abe77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:11.375955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717883a4-e5f3-40b7-bb1b-de3d2c7abe77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:11.385090 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717883a4-e5f3-40b7-bb1b-de3d2c7abe77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:11.386454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717883a4-e5f3-40b7-bb1b-de3d2c7abe77 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:11.396444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1293/2581] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:11.791969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:11.793395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:11.805991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e36d55-6d8d-4ee8-8bee-f625b1499f07 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:11.811429 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:11.812370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:11.812710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:11.812992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] PATH_INFO: `/v3/projects/db4b5f80260d4c58a745f3e4934dfa72` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:11.814357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db4b5f80260d4c58a745f3e4934dfa72)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:11.817235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:11.922063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db4b5f80260d4c58a745f3e4934dfa72', 'request_id': 'req-fb513465-90f5-43a7-b845-6027c96f9c68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.928220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db4b5f80260d4c58a745f3e4934dfa72', 'request_id': 'req-fb513465-90f5-43a7-b845-6027c96f9c68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.934095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db4b5f80260d4c58a745f3e4934dfa72', 'request_id': 'req-fb513465-90f5-43a7-b845-6027c96f9c68'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.995101 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] The token cache is being invalidate because project db4b5f80260d4c58a745f3e4934dfa72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:36:11.995414 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.996126 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-fb513465-90f5-43a7-b845-6027c96f9c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:11.997660 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1289/2582] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:36:11 2022] DELETE /identity/v3/projects/db4b5f80260d4c58a745f3e4934dfa72 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:12.002524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1ef0a-8872-43ee-97eb-824c040dfbd7 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.003052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1ef0a-8872-43ee-97eb-824c040dfbd7 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.003279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1ef0a-8872-43ee-97eb-824c040dfbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.022551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5cf1ef0a-8872-43ee-97eb-824c040dfbd7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:12.023823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5cf1ef0a-8872-43ee-97eb-824c040dfbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:12.058076 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:12.085303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1294/2583] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:12.093227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff35467-6a3f-4bb8-991c-1147ecb0f284 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.100933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff35467-6a3f-4bb8-991c-1147ecb0f284 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.100933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff35467-6a3f-4bb8-991c-1147ecb0f284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.112230 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-9ff35467-6a3f-4bb8-991c-1147ecb0f284 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:12.113958 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9ff35467-6a3f-4bb8-991c-1147ecb0f284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:12.122534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1290/2584] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:12.126486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1ef0a-8872-43ee-97eb-824c040dfbd7 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:12.136678 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1ef0a-8872-43ee-97eb-824c040dfbd7 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:12.141526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:12.142176 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.142459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.142733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.143452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:36:12.143785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:12.145394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:12.157412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1295/2585] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:36:12.161770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff35467-6a3f-4bb8-991c-1147ecb0f284 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:12.171073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff35467-6a3f-4bb8-991c-1147ecb0f284 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:12.176994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:12.177652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.177931 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.178209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.178929 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:12.180844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:12.275897 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1291/2586] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:36:12 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:36:12.280066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:12.290817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacd864-262c-408c-be74-b751c4836f7f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:12.347636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:12.348334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.348618 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.348895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.349598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:12.351537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:12.455378 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1296/2587] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:36:12 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:36:12.455966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:12.466522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e86f3-9441-4277-9106-94b6a3f10c54 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:12.471496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:12.472775 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.473053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.473329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.474052 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:12.475641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:12.486969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1292/2588] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:36:12.491714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:12.501933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29410065-6475-4d0f-8748-312fed91224f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:12.506687 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ee766d-3628-4a7a-be8e-ca821db1894f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:12.507349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee766d-3628-4a7a-be8e-ca821db1894f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.507638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee766d-3628-4a7a-be8e-ca821db1894f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.507955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ee766d-3628-4a7a-be8e-ca821db1894f admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.508744 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee766d-3628-4a7a-be8e-ca821db1894f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:12.510354 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ee766d-3628-4a7a-be8e-ca821db1894f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:12.520661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1297/2589] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:36:12.524736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:12.534655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f067a-ee51-40b8-b12c-9fdb79e1d647 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:12.540375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:12.541302 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.541584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.541858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] PATH_INFO: `/v3/projects/ef6dbe4fd15e40d2a4d188845462a63d/users/b44090073bce475b91dec4b58ac8cc83/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.544319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:12.558091 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef6dbe4fd15e40d2a4d188845462a63d, user_id=b44090073bce475b91dec4b58ac8cc83, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:12.560732 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:12.657292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1293/2590] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:36:12 2022] PUT /identity/v3/projects/ef6dbe4fd15e40d2a4d188845462a63d/users/b44090073bce475b91dec4b58ac8cc83/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:12.669185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0753feb-4f01-4408-9aa2-414c07d58325 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.669472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0753feb-4f01-4408-9aa2-414c07d58325 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.669746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0753feb-4f01-4408-9aa2-414c07d58325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.697267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-e0753feb-4f01-4408-9aa2-414c07d58325 None None] MFA Rules not processed for user `b44090073bce475b91dec4b58ac8cc83`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:12.698473 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-e0753feb-4f01-4408-9aa2-414c07d58325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:12.761849 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1298/2591] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:12.772913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:12.774636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:12.833308 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30003388-6944-4cdf-8a9d-c7c06dfa49cb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:12.836983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7a970c-37ae-483b-9440-5a3bc9fb8715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:12.837626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a970c-37ae-483b-9440-5a3bc9fb8715 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:12.837938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a970c-37ae-483b-9440-5a3bc9fb8715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:12.838219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7a970c-37ae-483b-9440-5a3bc9fb8715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:12.846563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a970c-37ae-483b-9440-5a3bc9fb8715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:12.848621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7a970c-37ae-483b-9440-5a3bc9fb8715 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:12.857476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1294/2592] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:15.770225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396692f6-ca72-40fd-baf6-90e240bf0dac None None] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:15.780436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396692f6-ca72-40fd-baf6-90e240bf0dac None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:15.780436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396692f6-ca72-40fd-baf6-90e240bf0dac None None] PATH_INFO: `/` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:15.780436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1299/2593] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:36:15.780762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9651-e660-4987-847f-8db9e18ce5f6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:15.780762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9651-e660-4987-847f-8db9e18ce5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:15.780762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9651-e660-4987-847f-8db9e18ce5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:15.798212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-78ba9651-e660-4987-847f-8db9e18ce5f6 None None] MFA Rules not processed for user `e757ff94b7474fd99a1c7221be19c64d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:15.799053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-78ba9651-e660-4987-847f-8db9e18ce5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:15.856537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1295/2594] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:36:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:36:15.868479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396692f6-ca72-40fd-baf6-90e240bf0dac None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:15.870231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-396692f6-ca72-40fd-baf6-90e240bf0dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:15.883336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396692f6-ca72-40fd-baf6-90e240bf0dac None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:15.886843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128aa173-65e3-4ac1-b3da-18206c724303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:15.887475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa173-65e3-4ac1-b3da-18206c724303 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:15.887782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa173-65e3-4ac1-b3da-18206c724303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:15.888063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128aa173-65e3-4ac1-b3da-18206c724303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:15.895453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128aa173-65e3-4ac1-b3da-18206c724303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:15.896913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128aa173-65e3-4ac1-b3da-18206c724303 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:15.904591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1300/2595] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:15.987536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac9da-972f-426d-ad12-9efb58a37eca None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:15.987923 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac9da-972f-426d-ad12-9efb58a37eca None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:15.988249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aac9da-972f-426d-ad12-9efb58a37eca None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:15.989653 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1296/2596] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:36:15.994937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc687-3bee-4b52-ab7e-b501272cf245 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:15.995252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc687-3bee-4b52-ab7e-b501272cf245 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:15.995528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc687-3bee-4b52-ab7e-b501272cf245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:16.014491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1c5bc687-3bee-4b52-ab7e-b501272cf245 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:16.015831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1c5bc687-3bee-4b52-ab7e-b501272cf245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:16.034093 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:16.084537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1301/2597] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:36:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:16.096172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac9da-972f-426d-ad12-9efb58a37eca None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:16.098489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f3aac9da-972f-426d-ad12-9efb58a37eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:16.111477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aac9da-972f-426d-ad12-9efb58a37eca None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:16.114998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b209a6-8895-4014-bd7d-ab7593ed562f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:16.115634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b209a6-8895-4014-bd7d-ab7593ed562f service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:16.115956 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b209a6-8895-4014-bd7d-ab7593ed562f service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:16.116236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b209a6-8895-4014-bd7d-ab7593ed562f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:16.126337 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b209a6-8895-4014-bd7d-ab7593ed562f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:16.127769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b209a6-8895-4014-bd7d-ab7593ed562f service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:16.139023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1297/2598] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:26.091212 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=21 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:26.133734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:40.072329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc687-3bee-4b52-ab7e-b501272cf245 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:40.139982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc687-3bee-4b52-ab7e-b501272cf245 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:40.144500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:40.145372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:40.145655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:40.145971 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] PATH_INFO: `/v3/users/b44090073bce475b91dec4b58ac8cc83` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:40.147159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b44090073bce475b91dec4b58ac8cc83)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:40.149128 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:40.356130 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b44090073bce475b91dec4b58ac8cc83'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:40.368376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b44090073bce475b91dec4b58ac8cc83'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:40.428091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b44090073bce475b91dec4b58ac8cc83'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:40.432108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b44090073bce475b91dec4b58ac8cc83'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:40.432108 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b44090073bce475b91dec4b58ac8cc83'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:40.443000 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1302/2599] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:36:40 2022] DELETE /identity/v3/users/b44090073bce475b91dec4b58ac8cc83 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:36:40.447474 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:40.447974 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:40.448100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:40.466147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:40.467409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:40.530908 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1298/2600] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:40.541286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:40.543024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:40.602695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cb14d-7f18-4e45-83dd-0e2b60d69389 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:40.606469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bdd36-054b-442a-878d-63c8fe04c0e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:40.607055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bdd36-054b-442a-878d-63c8fe04c0e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:40.607339 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bdd36-054b-442a-878d-63c8fe04c0e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:40.607613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bdd36-054b-442a-878d-63c8fe04c0e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:40.617605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bdd36-054b-442a-878d-63c8fe04c0e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:40.619247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0bdd36-054b-442a-878d-63c8fe04c0e3 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:40.629113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1303/2601] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:41.042412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:41.043728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:41.058118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95ac13-2ead-48f4-a08d-334b36307f3b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:41.062884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:41.063785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.064073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.064350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] PATH_INFO: `/v3/projects/ef6dbe4fd15e40d2a4d188845462a63d` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.066152 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef6dbe4fd15e40d2a4d188845462a63d)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:41.072734 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:41.203912 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef6dbe4fd15e40d2a4d188845462a63d', 'request_id': 'req-5605e2e4-a28f-4f49-98c8-e5e480c7034c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:41.210877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef6dbe4fd15e40d2a4d188845462a63d', 'request_id': 'req-5605e2e4-a28f-4f49-98c8-e5e480c7034c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:41.216451 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef6dbe4fd15e40d2a4d188845462a63d', 'request_id': 'req-5605e2e4-a28f-4f49-98c8-e5e480c7034c'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:41.331901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] The token cache is being invalidate because project ef6dbe4fd15e40d2a4d188845462a63d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:36:41.332195 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:41.332815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5605e2e4-a28f-4f49-98c8-e5e480c7034c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:41.334359 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1299/2602] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:36:41 2022] DELETE /identity/v3/projects/ef6dbe4fd15e40d2a4d188845462a63d => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:41.339513 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658cb37-3a82-48e7-95b7-bee5cc5bbb6c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.340049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658cb37-3a82-48e7-95b7-bee5cc5bbb6c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.340203 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658cb37-3a82-48e7-95b7-bee5cc5bbb6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.357922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-4658cb37-3a82-48e7-95b7-bee5cc5bbb6c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:41.359155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-4658cb37-3a82-48e7-95b7-bee5cc5bbb6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:41.376475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:41.418472 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1304/2603] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:41.426689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aa1d6-26ba-4afe-9a63-e11227839028 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.427482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aa1d6-26ba-4afe-9a63-e11227839028 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.427785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35aa1d6-26ba-4afe-9a63-e11227839028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.446616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b35aa1d6-26ba-4afe-9a63-e11227839028 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:41.447846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b35aa1d6-26ba-4afe-9a63-e11227839028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:41.455774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1300/2604] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:41.460033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658cb37-3a82-48e7-95b7-bee5cc5bbb6c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:41.470657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658cb37-3a82-48e7-95b7-bee5cc5bbb6c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:41.475363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:41.476054 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.476335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.476605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.477351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:36:41.477653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:41.479246 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:41.491454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1305/2605] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:36:41.495783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aa1d6-26ba-4afe-9a63-e11227839028 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:41.506080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35aa1d6-26ba-4afe-9a63-e11227839028 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:41.510776 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:41.511433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.511717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.512024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.512712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:41.514605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:41.584324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1301/2606] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:36:41 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:36:41.585225 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:41.595526 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fa611d-934e-4c62-9c51-4d36ffdda96b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:41.656658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:41.657346 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.657630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.658434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.658633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:41.660586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:41.728271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1306/2607] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:36:41 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:36:41.731546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:41.742930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e04c0e-08f7-47b5-81e8-29037ce51146 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:41.748216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:41.748889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.749173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.749452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.750296 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:41.751915 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:41.762823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1302/2608] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:36:41.766919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:41.777180 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b52c497-52ca-46ba-98cf-780e2571f150 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:41.781723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16538d1-eb0c-4661-8d69-b0aff5635f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:41.782377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16538d1-eb0c-4661-8d69-b0aff5635f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.782658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16538d1-eb0c-4661-8d69-b0aff5635f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.782935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16538d1-eb0c-4661-8d69-b0aff5635f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.783654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16538d1-eb0c-4661-8d69-b0aff5635f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:41.785413 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16538d1-eb0c-4661-8d69-b0aff5635f04 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:41.796551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1307/2609] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:36:41.800767 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:41.811251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a09cf5-773b-44db-aeab-f54903e128ab admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:41.815825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:41.816707 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.816993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.817270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] PATH_INFO: `/v3/projects/c19b969782614535ac76262b61efcb8a/users/63db7b41f6b544ceaf30bf54f4d45d99/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.831477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19b969782614535ac76262b61efcb8a, user_id=63db7b41f6b544ceaf30bf54f4d45d99, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:41.834145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:41.903893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1303/2610] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:36:41 2022] PUT /identity/v3/projects/c19b969782614535ac76262b61efcb8a/users/63db7b41f6b544ceaf30bf54f4d45d99/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:41.915892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75596-3aed-422c-b330-c7f1f9a04083 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:41.916188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75596-3aed-422c-b330-c7f1f9a04083 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:41.916465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca75596-3aed-422c-b330-c7f1f9a04083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:41.944389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-fca75596-3aed-422c-b330-c7f1f9a04083 None None] MFA Rules not processed for user `63db7b41f6b544ceaf30bf54f4d45d99`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:41.945638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fca75596-3aed-422c-b330-c7f1f9a04083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:42.009466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1308/2611] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:42.019951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:42.021664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:42.043953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:42.079696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204410f0-143f-4a32-943a-d83d7f8e9bb6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:42.083926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce29faf-f1ea-4fb7-b2ac-18d43d892f73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:42.084594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29faf-f1ea-4fb7-b2ac-18d43d892f73 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:42.084853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29faf-f1ea-4fb7-b2ac-18d43d892f73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:42.085129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce29faf-f1ea-4fb7-b2ac-18d43d892f73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:42.093983 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce29faf-f1ea-4fb7-b2ac-18d43d892f73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:42.095425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce29faf-f1ea-4fb7-b2ac-18d43d892f73 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:42.105227 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1304/2612] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:52.006355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:52.131837 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=17 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:53.057158 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca75596-3aed-422c-b330-c7f1f9a04083 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:53.122850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca75596-3aed-422c-b330-c7f1f9a04083 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:53.127493 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:53.128396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:53.131966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:53.131966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] PATH_INFO: `/v3/users/63db7b41f6b544ceaf30bf54f4d45d99` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:53.131966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63db7b41f6b544ceaf30bf54f4d45d99)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:53.132252 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:53.253670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63db7b41f6b544ceaf30bf54f4d45d99'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:53.265874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63db7b41f6b544ceaf30bf54f4d45d99'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:53.349162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63db7b41f6b544ceaf30bf54f4d45d99'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:53.349702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63db7b41f6b544ceaf30bf54f4d45d99'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:53.350302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63db7b41f6b544ceaf30bf54f4d45d99'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:53.363854 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1309/2613] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:36:53 2022] DELETE /identity/v3/users/63db7b41f6b544ceaf30bf54f4d45d99 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:53.368841 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:53.369345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:53.369476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:53.388348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:36:53.389555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:53.448943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1305/2614] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:36:53.458531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:53.459962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:53.519401 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f791d48e-ce1f-482d-b996-040467216064 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:53.523142 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:53.524037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:53.524164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:53.524359 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:53.534657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:53.536233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:53.546570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1310/2615] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:54.000600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:54.002015 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:54.014039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0f76a-2fb4-4d7a-bcfa-143c2c560c38 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:54.020507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:54.020507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:54.020507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:54.020798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] PATH_INFO: `/v3/projects/c19b969782614535ac76262b61efcb8a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:54.022055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19b969782614535ac76262b61efcb8a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:54.025688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:54.186370 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19b969782614535ac76262b61efcb8a', 'request_id': 'req-999da40e-6c1b-40d0-a411-f885001ead19'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:54.193327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19b969782614535ac76262b61efcb8a', 'request_id': 'req-999da40e-6c1b-40d0-a411-f885001ead19'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:54.199053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19b969782614535ac76262b61efcb8a', 'request_id': 'req-999da40e-6c1b-40d0-a411-f885001ead19'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:54.272835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] The token cache is being invalidate because project c19b969782614535ac76262b61efcb8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:36:54.283099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:54.283099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:36:54.283099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1306/2616] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:36:53 2022] DELETE /identity/v3/projects/c19b969782614535ac76262b61efcb8a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:36:56.654920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:56.664068 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7bca-f0f1-4185-90d3-c9b69a5c075f service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:56.683454 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:36:56.719080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:56.719735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:56.720047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:56.720324 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:56.722289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:56.796922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:56.798570 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:56.811383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1311/2617] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:36:57.010441 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:36:57.012560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:57.070596 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999da40e-6c1b-40d0-a411-f885001ead19 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:36:57.074901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6eb6c9-f394-4dd1-bc92-4bbd694ffd78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:36:57.075553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eb6c9-f394-4dd1-bc92-4bbd694ffd78 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:36:57.075870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eb6c9-f394-4dd1-bc92-4bbd694ffd78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:36:57.076149 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6eb6c9-f394-4dd1-bc92-4bbd694ffd78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:36:57.077942 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1d6eb6c9-f394-4dd1-bc92-4bbd694ffd78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:36:57.091430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6eb6c9-f394-4dd1-bc92-4bbd694ffd78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:36:57.092886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6eb6c9-f394-4dd1-bc92-4bbd694ffd78 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:36:57.104508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1307/2618] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:36:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:06.145056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:06.146437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:06.160780 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924aa32-9061-4f6e-ae35-c1966812c48e service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:06.166394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:06.167045 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:06.167331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:06.167609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:06.170016 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:06.260964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:06.263178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:06.282850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1312/2619] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:07.100077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:37:16.080173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:16.080756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:16.080756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:16.129843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] MFA Rules not processed for user `6214705ae31e459891f7a215f45a4cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:37:16.131140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:16.195136 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1308/2620] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:37:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:37:16.206591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:16.217591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548190a2-e22b-44ad-b4d5-75ae868ad97b service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:16.222988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:16.223621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:16.223932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:16.224215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:16.225981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:16.236910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:16.239723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:16.253785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1313/2621] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:16.548270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:16.549678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:16.561373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d665-d1bb-47c2-b9a7-1cd1067b6594 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:16.564889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881292ec-edc6-4e98-b7d4-4f30a2f9e6ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:16.565551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881292ec-edc6-4e98-b7d4-4f30a2f9e6ed service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:16.565830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881292ec-edc6-4e98-b7d4-4f30a2f9e6ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:16.566103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881292ec-edc6-4e98-b7d4-4f30a2f9e6ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:16.568011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-881292ec-edc6-4e98-b7d4-4f30a2f9e6ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:16.577165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881292ec-edc6-4e98-b7d4-4f30a2f9e6ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:16.578567 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881292ec-edc6-4e98-b7d4-4f30a2f9e6ed service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:16.586909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1309/2622] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:17.020705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:17.022125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:17.050048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:37:17.106649 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7ce39-e931-4329-9c14-cd1d093695f2 service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:17.112042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:17.112708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:17.112994 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:17.114301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:17.116774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:17.130238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:17.132706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:17.144625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1314/2623] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:17.543780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:17.544292 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:17.544456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:17.563062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:37:17.564289 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:17.622338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1310/2624] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:37:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:37:17.776432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:17.778460 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:17.848617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45b1a9-9e56-4678-b9c9-cf1cfea8fa06 service placement] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:17.853786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:17.854425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:17.854704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:17.855192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:17.863992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:17.866831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:17.875415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1315/2625] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:37:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:37:27.614614 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=72 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:37:27.875456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=55 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:37:29.875325 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:29.876900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:29.892407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6e025-e718-4207-a8b4-6058f606e020 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:29.896327 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a527c-a5e3-4e84-adb5-c10f41935cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:29.897046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a527c-a5e3-4e84-adb5-c10f41935cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:29.897397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a527c-a5e3-4e84-adb5-c10f41935cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:29.898492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a527c-a5e3-4e84-adb5-c10f41935cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:29.900825 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d26a527c-a5e3-4e84-adb5-c10f41935cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:29.918313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a527c-a5e3-4e84-adb5-c10f41935cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:29.919914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26a527c-a5e3-4e84-adb5-c10f41935cc8 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:29.929545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1311/2626] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:39.465164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:39.466701 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:39.482201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee422c38-a161-41b6-b453-ca8c071b7084 service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:39.488048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884649a7-cb1d-4957-93d1-80bf2b27321e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:39.488707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884649a7-cb1d-4957-93d1-80bf2b27321e service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:39.489023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884649a7-cb1d-4957-93d1-80bf2b27321e service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:39.489304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884649a7-cb1d-4957-93d1-80bf2b27321e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:39.491421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-884649a7-cb1d-4957-93d1-80bf2b27321e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:39.505043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884649a7-cb1d-4957-93d1-80bf2b27321e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:39.506659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884649a7-cb1d-4957-93d1-80bf2b27321e service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:39.518657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1316/2627] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:39.929687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=6 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:37:40.064550 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c0adb-81f9-44c5-9ae9-bb7932d0aa7a None None] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:40.065886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c0adb-81f9-44c5-9ae9-bb7932d0aa7a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:40.066341 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c0adb-81f9-44c5-9ae9-bb7932d0aa7a None None] PATH_INFO: `/` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:40.068860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1312/2628] 10.222.0.42 () {54 vars in 921 bytes} [Thu May 26 21:37:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 21:37:40.075209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff00451-39dd-409b-ab1e-551d91c8b493 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:40.075209 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff00451-39dd-409b-ab1e-551d91c8b493 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:40.076912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff00451-39dd-409b-ab1e-551d91c8b493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:40.106119 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-7ff00451-39dd-409b-ab1e-551d91c8b493 None None] MFA Rules not processed for user `eefd00457d354271867932d336d7f69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:37:40.107977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-7ff00451-39dd-409b-ab1e-551d91c8b493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:40.121453 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1317/2629] 10.222.0.42 () {58 vars in 1018 bytes} [Thu May 26 21:37:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:37:40.138491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c0adb-81f9-44c5-9ae9-bb7932d0aa7a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:40.144569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5b0c0adb-81f9-44c5-9ae9-bb7932d0aa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:40.213715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c0adb-81f9-44c5-9ae9-bb7932d0aa7a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:40.217427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:40.218084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:40.218360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:40.218632 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:40.229432 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:40.231390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:40.242855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1313/2630] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:40.479579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acdb32-69e7-4ee3-8095-e0aba751c85e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:40.480670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acdb32-69e7-4ee3-8095-e0aba751c85e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:40.480670 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acdb32-69e7-4ee3-8095-e0aba751c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:40.505615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-82acdb32-69e7-4ee3-8095-e0aba751c85e None None] MFA Rules not processed for user `88575c0bd9db4fa8bc9bd511c0ac37dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:37:40.506898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-82acdb32-69e7-4ee3-8095-e0aba751c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:40.581136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1318/2631] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:37:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:37:40.910502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:40.920621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec764503-1a57-4f82-b38c-e169617481cf service glance] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:40.924267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:40.924937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:40.925214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:40.925493 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:40.927269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:40.936849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:40.938257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:40.946233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1314/2632] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:37:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:37:42.752509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f26de2-3ea9-465a-a074-b1cb4638ddfa None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:42.753285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f26de2-3ea9-465a-a074-b1cb4638ddfa None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:42.753285 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f26de2-3ea9-465a-a074-b1cb4638ddfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:42.771101 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-79f26de2-3ea9-465a-a074-b1cb4638ddfa None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:37:42.772329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-79f26de2-3ea9-465a-a074-b1cb4638ddfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:37:42.780120 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1319/2633] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:37:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:37:42.793217 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:37:42.806044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ace0f-f929-4c2f-842a-e94e2b8138c2 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:37:42.811876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:37:42.812597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:37:42.812880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:37:42.813169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:37:42.822127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:37:42.823723 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:37:42.832096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1315/2634] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:37:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:37:52.765535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=37 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:37:52.828763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=38 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:05.711002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16363a2b-ef0e-40da-92c5-fa4aff870263 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:05.711411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16363a2b-ef0e-40da-92c5-fa4aff870263 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:05.711585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16363a2b-ef0e-40da-92c5-fa4aff870263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:05.729636 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-16363a2b-ef0e-40da-92c5-fa4aff870263 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:38:05.730892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-16363a2b-ef0e-40da-92c5-fa4aff870263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:05.738032 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1320/2635] 10.222.0.42 () {58 vars in 967 bytes} [Thu May 26 21:38:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:38:05.750007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:05.761343 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd20fbb0-c55c-4dab-8515-f88589a88bc8 service swift] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:05.766498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:05.767127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:05.767406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:05.767681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:05.775762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:05.778034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:05.786079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1316/2636] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:38:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:38:14.336123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16363a2b-ef0e-40da-92c5-fa4aff870263 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:14.349221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16363a2b-ef0e-40da-92c5-fa4aff870263 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:14.354643 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:14.355892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:14.359447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:14.359447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:14.359447 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:14.370149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:14.371772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:14.382152 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1321/2637] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:38:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:38:15.782838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=6 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:17.490039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:17.499940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0571c974-96e5-431d-81c6-d07d1f56bda9 service swift] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:17.503478 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:17.504155 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:17.504436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:17.504708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:17.506467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:17.570383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:17.571857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:17.581532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1317/2638] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:38:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:38:24.377775 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=9 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:27.577124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=30 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:30.636024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:30.662702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d702e120-1815-4c9d-82ab-c769d3c1c09b service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:30.666677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:30.667316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:30.667637 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:30.667966 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:30.670211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:30.761258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:30.763436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:30.773125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1322/2639] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:38:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:38:38.311551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:38.353069 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3aac19-957b-4d60-9669-097cfbbf7b38 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:38.357848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:38.363299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:38.363299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:38.363299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:38.363299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:38.460918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:38.462356 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:38.482378 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1318/2640] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:38:40.768560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=43 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:46.147961 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:46.149740 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:46.165664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3936333e-7aa4-4f9a-be51-64edf4630b2b service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:46.170591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:46.171547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:46.171870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:46.172147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] PATH_INFO: `/v3/users/38166757db224367bb8208f6366a4184` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:46.173334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38166757db224367bb8208f6366a4184)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:46.175353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:46.357545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38166757db224367bb8208f6366a4184'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.370622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38166757db224367bb8208f6366a4184'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.449781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38166757db224367bb8208f6366a4184'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.450421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38166757db224367bb8208f6366a4184'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.451661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38166757db224367bb8208f6366a4184'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.464575 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1323/2641] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:38:46 2022] DELETE /identity/v3/users/38166757db224367bb8208f6366a4184 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:38:46.719994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:46.730077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429ef37-08cc-40ed-b5f4-1c3795fe65ea service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:46.789179 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:46.790041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:46.790355 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:46.790634 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] PATH_INFO: `/v3/projects/77c3cd3a54424befb4b424e6de1f7870` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:46.791851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77c3cd3a54424befb4b424e6de1f7870)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:46.793749 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:46.886763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:46.895485 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77c3cd3a54424befb4b424e6de1f7870', 'request_id': 'req-6188ca44-5418-412b-a3d9-4930596566e4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.901609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77c3cd3a54424befb4b424e6de1f7870', 'request_id': 'req-6188ca44-5418-412b-a3d9-4930596566e4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.906982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77c3cd3a54424befb4b424e6de1f7870', 'request_id': 'req-6188ca44-5418-412b-a3d9-4930596566e4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.974468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] The token cache is being invalidate because project 77c3cd3a54424befb4b424e6de1f7870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:38:46.974759 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.975396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:46.977104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1319/2642] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:38:46 2022] DELETE /identity/v3/projects/77c3cd3a54424befb4b424e6de1f7870 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:38:46.981022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:46.982983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:46.996375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd86a0-b8a8-4aed-b7a1-56df3d26a06f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:47.057067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:47.057948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:47.058226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:47.058499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] PATH_INFO: `/v3/users/5316267bacc147ed9fa67e0a9dc16338` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:47.059770 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5316267bacc147ed9fa67e0a9dc16338)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:47.061751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:47.238598 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5316267bacc147ed9fa67e0a9dc16338'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.260024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5316267bacc147ed9fa67e0a9dc16338'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.341684 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5316267bacc147ed9fa67e0a9dc16338'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.342310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5316267bacc147ed9fa67e0a9dc16338'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.342922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5316267bacc147ed9fa67e0a9dc16338'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.345506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:47.357632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1324/2643] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:38:46 2022] DELETE /identity/v3/users/5316267bacc147ed9fa67e0a9dc16338 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:38:47.628500 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:38:47.629933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:38:47.644972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188ca44-5418-412b-a3d9-4930596566e4 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:38:47.702703 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:38:47.703585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:38:47.703886 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:38:47.704160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] PATH_INFO: `/v3/projects/7ddb778984b64029ae40ad6be7f1bf20` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:38:47.705364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ddb778984b64029ae40ad6be7f1bf20)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:38:47.707264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:38:47.854518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ddb778984b64029ae40ad6be7f1bf20', 'request_id': 'req-c581efa8-990f-4801-be72-544fe3c51aaf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.861239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ddb778984b64029ae40ad6be7f1bf20', 'request_id': 'req-c581efa8-990f-4801-be72-544fe3c51aaf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.867678 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ddb778984b64029ae40ad6be7f1bf20', 'request_id': 'req-c581efa8-990f-4801-be72-544fe3c51aaf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.958174 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] The token cache is being invalidate because project 7ddb778984b64029ae40ad6be7f1bf20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:38:47.958464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.959083 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:38:47.960655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1320/2644] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:38:47 2022] DELETE /identity/v3/projects/7ddb778984b64029ae40ad6be7f1bf20 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:38:57.449375 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:38:57.954853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:23.427039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:23.428507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:23.499373 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6f790-849a-4e3d-973f-e1889769d25d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:23.549178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:23.549866 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:23.550154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:23.550436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:23.552423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:23.615302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:23.616728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:23.627148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1325/2645] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:27.410849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:27.412560 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:27.427125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581efa8-990f-4801-be72-544fe3c51aaf admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:27.432457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:27.433430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:27.433777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:27.434127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] PATH_INFO: `/v3/users/9b94748ea78342629281f2360a54bee4` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:27.435998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b94748ea78342629281f2360a54bee4)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:27.437930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:27.652048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b94748ea78342629281f2360a54bee4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:27.665167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b94748ea78342629281f2360a54bee4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:27.745685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b94748ea78342629281f2360a54bee4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:27.746332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b94748ea78342629281f2360a54bee4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:27.746937 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-cced90e4-5be4-4508-a98c-339ad77f9110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b94748ea78342629281f2360a54bee4'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:27.759803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1321/2646] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:39:27 2022] DELETE /identity/v3/users/9b94748ea78342629281f2360a54bee4 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:27.999536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:28.019595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc688d82-c031-4ccd-8ef8-1555c08808de service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:28.081668 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=100 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:28.088851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:28.091457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.091457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.091457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] PATH_INFO: `/v3/projects/c5f69588fe354d20b58594919deb4110` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.092491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5f69588fe354d20b58594919deb4110)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:28.095310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:28.237364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5f69588fe354d20b58594919deb4110', 'request_id': 'req-64b2910b-2dff-4000-8fec-5d6b165f60ff'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:28.244383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5f69588fe354d20b58594919deb4110', 'request_id': 'req-64b2910b-2dff-4000-8fec-5d6b165f60ff'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:28.250868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5f69588fe354d20b58594919deb4110', 'request_id': 'req-64b2910b-2dff-4000-8fec-5d6b165f60ff'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:28.340669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] The token cache is being invalidate because project c5f69588fe354d20b58594919deb4110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:39:28.340945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:28.341617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-64b2910b-2dff-4000-8fec-5d6b165f60ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:28.343236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1326/2647] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:39:27 2022] DELETE /identity/v3/projects/c5f69588fe354d20b58594919deb4110 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:28.348972 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970a62-19e6-4c31-8b68-b03a69cb429f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.349603 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970a62-19e6-4c31-8b68-b03a69cb429f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.349999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff970a62-19e6-4c31-8b68-b03a69cb429f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.407820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-ff970a62-19e6-4c31-8b68-b03a69cb429f None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:28.409041 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-ff970a62-19e6-4c31-8b68-b03a69cb429f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:28.467868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1322/2648] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:28.475666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5fbbc-5505-4f9b-a15a-a7c10606fa4d None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.475975 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5fbbc-5505-4f9b-a15a-a7c10606fa4d None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.476251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5fbbc-5505-4f9b-a15a-a7c10606fa4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.496547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f4d5fbbc-5505-4f9b-a15a-a7c10606fa4d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:28.498323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f4d5fbbc-5505-4f9b-a15a-a7c10606fa4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:28.506395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1327/2649] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:28.510743 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970a62-19e6-4c31-8b68-b03a69cb429f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:28.521037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff970a62-19e6-4c31-8b68-b03a69cb429f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:28.525539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:28.526213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.526502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.526782 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.527524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:39:28.527900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:28.529626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:28.542039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1323/2650] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:39:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:39:28.547399 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5fbbc-5505-4f9b-a15a-a7c10606fa4d None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:28.557259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5fbbc-5505-4f9b-a15a-a7c10606fa4d None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:28.562717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:28.563394 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.563713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.564014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.564720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:28.566647 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:28.684793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1328/2651] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:39:28 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:39:28.689422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:28.699650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82a8367-7227-41a1-bdee-aa9106bdf35e admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:28.725884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:28.755860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:28.756545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.756827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.757108 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.757847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:28.759680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:28.900287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1324/2652] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:39:28 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:28.904215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:28.914641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23f113-144a-4d4e-aa54-dfce27108c3a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:28.919776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:28.920651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.921036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.921414 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.922351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:28.924082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:28.934662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1329/2653] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:28.939129 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:28.949144 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8df32-75f8-401b-901d-e99ee8358318 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:28.954438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ea66f-1b67-494d-973d-b61580c146ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:28.955406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ea66f-1b67-494d-973d-b61580c146ee admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.955406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ea66f-1b67-494d-973d-b61580c146ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.955605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ea66f-1b67-494d-973d-b61580c146ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:28.956371 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ea66f-1b67-494d-973d-b61580c146ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:28.957945 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ea66f-1b67-494d-973d-b61580c146ee admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:28.967978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1325/2654] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:28.972039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:28.981995 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c5a63-a257-4578-9be8-c7bf23d84fc1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:28.986567 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:28.989057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:28.989057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:28.989057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] PATH_INFO: `/v3/projects/fe887b2709bc446c8198c3d141c353af/users/63e98f6491344656a94e199575494def/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:29.003704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe887b2709bc446c8198c3d141c353af, user_id=63e98f6491344656a94e199575494def, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:29.006449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:29.125496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1330/2655] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:28 2022] PUT /identity/v3/projects/fe887b2709bc446c8198c3d141c353af/users/63e98f6491344656a94e199575494def/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:39:29.130409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:29.130699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:29.131080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:29.155209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:29.156440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:29.216409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1326/2656] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:29.226565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:29.228302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:29.284279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e6c02-50c4-488e-8038-d309bf3c0205 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:29.288067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f40a7-30ba-4969-9a37-dce199bb5513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:29.288746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f40a7-30ba-4969-9a37-dce199bb5513 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:29.289027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f40a7-30ba-4969-9a37-dce199bb5513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:29.289302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f40a7-30ba-4969-9a37-dce199bb5513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:29.299389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f40a7-30ba-4969-9a37-dce199bb5513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:29.300879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f40a7-30ba-4969-9a37-dce199bb5513 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:29.311071 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1331/2657] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:31.689094 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:31.690565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:31.705590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c1f84-5405-4acb-bf8b-a026517b2442 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:31.709505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:31.710361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:31.710646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:31.710924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:31.712848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:31.725858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:31.727304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:31.744089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1327/2658] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:35.697271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:35.697739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:35.697976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:35.738965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] MFA Rules not processed for user `63e98f6491344656a94e199575494def`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:35.740286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:35.754463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:35.824690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1332/2659] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:35.840007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:35.842010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:35.935625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:35.935625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:35.978294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10cfc-b0ec-4fff-b2ab-33d838bec9c3 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:35.987392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:35.990365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:35.990365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:35.990365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:36.002118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:36.007549 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:36.010019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:36.023536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1328/2660] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:36.035105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9602ada-a9ca-4202-a020-749860ff432a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:36.041624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:36.042534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:36.042827 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:36.043105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:36.045297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:36.058308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:36.059959 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:36.086295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1333/2661] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:39:36.087526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:36.088990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:36.106579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fedc6-7162-49f6-9621-1e74315fc8b2 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:36.112465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:36.116504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:36.116504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:36.116504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] PATH_INFO: `/v3/users/6214705ae31e459891f7a215f45a4cb2` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:36.116504 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6214705ae31e459891f7a215f45a4cb2)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:36.121778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:36.353085 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6214705ae31e459891f7a215f45a4cb2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:36.380236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6214705ae31e459891f7a215f45a4cb2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:36.441684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6214705ae31e459891f7a215f45a4cb2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:36.442319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6214705ae31e459891f7a215f45a4cb2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:36.443120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-5a744d36-cf99-4fe8-b3bb-18d3524fa070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6214705ae31e459891f7a215f45a4cb2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:36.463507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1329/2662] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:39:36 2022] DELETE /identity/v3/users/6214705ae31e459891f7a215f45a4cb2 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:36.691104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:36.706540 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7980acc-ac2c-4dc4-8a23-246a4aab35fa service placement] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:36.774508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:36.776385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:36.776385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:36.776385 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] PATH_INFO: `/v3/projects/3969075d983246cf8c4c7a45a57d5338` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:36.777765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] RBAC: Authorizing `identity:delete_project(project_id=3969075d983246cf8c4c7a45a57d5338)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:36.779880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:36.900751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:36.903751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3969075d983246cf8c4c7a45a57d5338', 'request_id': 'req-8f62a390-16cc-46d9-9054-fc23c72d3dde'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:36.909981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3969075d983246cf8c4c7a45a57d5338', 'request_id': 'req-8f62a390-16cc-46d9-9054-fc23c72d3dde'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:36.916014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3969075d983246cf8c4c7a45a57d5338', 'request_id': 'req-8f62a390-16cc-46d9-9054-fc23c72d3dde'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:37.015710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] The token cache is being invalidate because project 3969075d983246cf8c4c7a45a57d5338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:39:37.016216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:37.017580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8f62a390-16cc-46d9-9054-fc23c72d3dde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:37.020245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1334/2663] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:39:36 2022] DELETE /identity/v3/projects/3969075d983246cf8c4c7a45a57d5338 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:37.025397 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1397fc3-16f1-4bcd-b833-4b46075d7570 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.025960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1397fc3-16f1-4bcd-b833-4b46075d7570 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.025960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1397fc3-16f1-4bcd-b833-4b46075d7570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.047978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c1397fc3-16f1-4bcd-b833-4b46075d7570 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:37.049508 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c1397fc3-16f1-4bcd-b833-4b46075d7570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:37.121617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1330/2664] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:37.130843 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d51ada-e17a-423f-8307-5cd45d97a66e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.131122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d51ada-e17a-423f-8307-5cd45d97a66e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.138958 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d51ada-e17a-423f-8307-5cd45d97a66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.157561 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-b3d51ada-e17a-423f-8307-5cd45d97a66e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:37.158785 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b3d51ada-e17a-423f-8307-5cd45d97a66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:37.167059 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1335/2665] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:37.171636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1397fc3-16f1-4bcd-b833-4b46075d7570 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:37.181329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1397fc3-16f1-4bcd-b833-4b46075d7570 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:37.186992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:37.187662 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.187987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.188265 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.188997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:39:37.189294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:37.190961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:37.203822 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1331/2666] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:39:37.208300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d51ada-e17a-423f-8307-5cd45d97a66e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:37.219039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d51ada-e17a-423f-8307-5cd45d97a66e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:37.227553 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:37.228341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.228691 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.229168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.230706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:37.232560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:37.310653 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1336/2667] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:39:37 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:37.312559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:37.325096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded65d2e-5024-45d3-90eb-bb4b99539cdb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:37.377440 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:37.392008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:37.392008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.392008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.392008 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.392426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:37.393658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:37.506665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1332/2668] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:39:37 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:37.510851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:37.526379 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac4073-1dfa-4e7d-9df4-b3d757313f8d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:37.532185 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:37.532878 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.533163 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.533444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.534218 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:37.535875 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:37.551196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1337/2669] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:37.556056 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:37.572028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf302-3c3e-4594-b12c-4a1df5367c97 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:37.578514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377cad4-a566-41f6-bdac-feb362ddd9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:37.579926 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377cad4-a566-41f6-bdac-feb362ddd9cd admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.580399 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377cad4-a566-41f6-bdac-feb362ddd9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.580830 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377cad4-a566-41f6-bdac-feb362ddd9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.581750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377cad4-a566-41f6-bdac-feb362ddd9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:37.586204 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377cad4-a566-41f6-bdac-feb362ddd9cd admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:37.601509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1333/2670] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:37.608662 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:37.619779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74568672-0bbd-4951-b9aa-a5be7ea03051 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:37.625632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:37.629948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.629948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.629948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] PATH_INFO: `/v3/projects/112131d351a3403b811018bc4e233703/users/90b7110268ea4c6e86418971763f0a72/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.645040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=112131d351a3403b811018bc4e233703, user_id=90b7110268ea4c6e86418971763f0a72, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:37.647889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:37.741259 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1338/2671] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:37 2022] PUT /identity/v3/projects/112131d351a3403b811018bc4e233703/users/90b7110268ea4c6e86418971763f0a72/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:37.754301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bb66d-2fb8-44e7-86ef-6f44d82b7b73 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:37.754610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bb66d-2fb8-44e7-86ef-6f44d82b7b73 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:37.754891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4bb66d-2fb8-44e7-86ef-6f44d82b7b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:37.789268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7b4bb66d-2fb8-44e7-86ef-6f44d82b7b73 None None] MFA Rules not processed for user `90b7110268ea4c6e86418971763f0a72`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:37.790509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7b4bb66d-2fb8-44e7-86ef-6f44d82b7b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:37.885126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:37.885348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1334/2672] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:38.012979 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028017f7-b80b-4c80-a75d-e2a3dd769e61 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:38.018870 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:38.024544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.024544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.024544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.024544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:38.024544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:38.153506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1339/2673] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:39:37 2022] POST /identity/v3/projects => generated 369 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:38.158818 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bb66d-2fb8-44e7-86ef-6f44d82b7b73 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:38.174816 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4bb66d-2fb8-44e7-86ef-6f44d82b7b73 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:38.241368 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:38.242205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.242589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.242970 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.243787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:38.246012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:38.351432 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1335/2674] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:39:38 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:38.355500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:38.366786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eb5df-ff02-4c95-8cf0-365631c411a8 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:38.371952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:38.372766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.373241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.373393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.374741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:38.376709 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:38.397344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1340/2675] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:38.398192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:38.410165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b41534-a782-4083-8e74-9015bc198e17 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:38.416125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab2fe7-7b05-481f-b0d5-b7db94aaa5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:38.417712 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab2fe7-7b05-481f-b0d5-b7db94aaa5f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.418250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab2fe7-7b05-481f-b0d5-b7db94aaa5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.419192 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab2fe7-7b05-481f-b0d5-b7db94aaa5f6 admin admin] PATH_INFO: `/v3/projects/e53c2cdde3ae41f9869a6eb9219db3f1/users/e39959b2382647909bcb3eb4ac15671d/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.465386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab2fe7-7b05-481f-b0d5-b7db94aaa5f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53c2cdde3ae41f9869a6eb9219db3f1, user_id=e39959b2382647909bcb3eb4ac15671d, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:38.468687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab2fe7-7b05-481f-b0d5-b7db94aaa5f6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:38.574913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1336/2676] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:38 2022] PUT /identity/v3/projects/e53c2cdde3ae41f9869a6eb9219db3f1/users/e39959b2382647909bcb3eb4ac15671d/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:38.579066 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:38.590894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74c9e5-f6bd-43de-a9f4-1938d35cfbb3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:38.640277 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.640571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.640854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.645766 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:38.659920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] MFA Rules not processed for user `63e98f6491344656a94e199575494def`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:38.661157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:38.662495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:38.663177 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.663449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.663726 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.664486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:38.672181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:38.683609 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1341/2677] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:38.684924 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:38.687328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:38.696310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c750e5-bf26-43ec-958d-3c52cd261e87 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:38.701895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9dae0-23e7-450b-8357-bd0036e890ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:38.702887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9dae0-23e7-450b-8357-bd0036e890ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.703238 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9dae0-23e7-450b-8357-bd0036e890ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.703586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9dae0-23e7-450b-8357-bd0036e890ad admin admin] PATH_INFO: `/v3/projects/e53c2cdde3ae41f9869a6eb9219db3f1/users/e39959b2382647909bcb3eb4ac15671d/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.708253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9dae0-23e7-450b-8357-bd0036e890ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e53c2cdde3ae41f9869a6eb9219db3f1, user_id=e39959b2382647909bcb3eb4ac15671d, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:38.712094 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9dae0-23e7-450b-8357-bd0036e890ad admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:38.726199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1342/2678] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:38 2022] PUT /identity/v3/projects/e53c2cdde3ae41f9869a6eb9219db3f1/users/e39959b2382647909bcb3eb4ac15671d/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:38.734328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1337/2679] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:39:38.737075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902da4-c68e-413e-bddc-231e8ce8b111 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:38.737367 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902da4-c68e-413e-bddc-231e8ce8b111 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:38.737645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902da4-c68e-413e-bddc-231e8ce8b111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:38.768929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3c902da4-c68e-413e-bddc-231e8ce8b111 None None] MFA Rules not processed for user `e39959b2382647909bcb3eb4ac15671d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:38.769411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3c902da4-c68e-413e-bddc-231e8ce8b111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:38.847919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1343/2680] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:38.856329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:38.858096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:38.963411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7888021-2513-4134-b41d-52a1de2d965f None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:39.029491 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850de43-cbd6-4ba1-b036-bb19cefcebdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:39.030436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850de43-cbd6-4ba1-b036-bb19cefcebdb service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:39.030917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850de43-cbd6-4ba1-b036-bb19cefcebdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:39.031427 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850de43-cbd6-4ba1-b036-bb19cefcebdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:39.042539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850de43-cbd6-4ba1-b036-bb19cefcebdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:39.044329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850de43-cbd6-4ba1-b036-bb19cefcebdb service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:39.055844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1338/2681] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:42.044879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902da4-c68e-413e-bddc-231e8ce8b111 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:42.117507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902da4-c68e-413e-bddc-231e8ce8b111 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:42.124215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:42.125641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:42.125922 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:42.126196 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] PATH_INFO: `/v3/users/90b7110268ea4c6e86418971763f0a72` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:42.127797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b7110268ea4c6e86418971763f0a72)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:42.129623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:42.373518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b7110268ea4c6e86418971763f0a72'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:42.381580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:42.392812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b7110268ea4c6e86418971763f0a72'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:42.511228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b7110268ea4c6e86418971763f0a72'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:42.512015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b7110268ea4c6e86418971763f0a72'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:42.512704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b7110268ea4c6e86418971763f0a72'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:42.528192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1344/2682] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:39:42 2022] DELETE /identity/v3/users/90b7110268ea4c6e86418971763f0a72 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:42.533073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:42.533570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:42.533695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:42.551673 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:42.558358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:42.633528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1339/2683] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:42.643774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:42.645403 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:42.706664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efdfb8-2e02-4150-a279-4c6825b291fc admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:42.711232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:42.712300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:42.713254 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:42.716436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:42.727468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:42.728983 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:42.739797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1345/2684] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:43.191646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:43.194100 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:43.206383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a32964-1cc5-4c88-8648-98f11e5913ef None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:43.211590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:43.213133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:43.213558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:43.214259 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] PATH_INFO: `/v3/projects/112131d351a3403b811018bc4e233703` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:43.215029 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=112131d351a3403b811018bc4e233703)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:43.217123 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:43.224383 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=100 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:43.368375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112131d351a3403b811018bc4e233703', 'request_id': 'req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.377010 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112131d351a3403b811018bc4e233703', 'request_id': 'req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.389893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112131d351a3403b811018bc4e233703', 'request_id': 'req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.467720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] The token cache is being invalidate because project 112131d351a3403b811018bc4e233703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:39:43.469072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.469842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.471910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1340/2685] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:39:43 2022] DELETE /identity/v3/projects/112131d351a3403b811018bc4e233703 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:43.475797 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:43.487868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6309a57-680c-48c3-b4b5-961517342068 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:43.549759 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:43.550628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:43.550912 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:43.551184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] PATH_INFO: `/v3/users/e39959b2382647909bcb3eb4ac15671d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:43.552620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e39959b2382647909bcb3eb4ac15671d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:43.554548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:43.853534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e39959b2382647909bcb3eb4ac15671d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.868442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e39959b2382647909bcb3eb4ac15671d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.951427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e39959b2382647909bcb3eb4ac15671d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.952175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e39959b2382647909bcb3eb4ac15671d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.952881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-3022e1bb-306a-4881-a033-af0f2925ebab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e39959b2382647909bcb3eb4ac15671d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:43.959088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:43.968588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1346/2686] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:39:43 2022] DELETE /identity/v3/users/e39959b2382647909bcb3eb4ac15671d => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:44.445058 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:44.446727 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:44.461480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2ff00c-0abd-445a-8201-a26e8eb7dfcf admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:44.534077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:44.534993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:44.535272 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:44.535547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] PATH_INFO: `/v3/projects/e53c2cdde3ae41f9869a6eb9219db3f1` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:44.536787 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e53c2cdde3ae41f9869a6eb9219db3f1)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:44.538693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:44.653426 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e53c2cdde3ae41f9869a6eb9219db3f1', 'request_id': 'req-8930709b-8f46-419c-93eb-a0483ee47f90'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:44.659694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e53c2cdde3ae41f9869a6eb9219db3f1', 'request_id': 'req-8930709b-8f46-419c-93eb-a0483ee47f90'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:44.665208 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e53c2cdde3ae41f9869a6eb9219db3f1', 'request_id': 'req-8930709b-8f46-419c-93eb-a0483ee47f90'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:44.739643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] The token cache is being invalidate because project e53c2cdde3ae41f9869a6eb9219db3f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:39:44.739969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:44.740629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8930709b-8f46-419c-93eb-a0483ee47f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:44.742107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1341/2687] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:39:44 2022] DELETE /identity/v3/projects/e53c2cdde3ae41f9869a6eb9219db3f1 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:44.747001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d269c-eb23-43ec-8f90-1c7b39a79967 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:44.747507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d269c-eb23-43ec-8f90-1c7b39a79967 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:44.747607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d269c-eb23-43ec-8f90-1c7b39a79967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:44.765903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-a52d269c-eb23-43ec-8f90-1c7b39a79967 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:44.767221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a52d269c-eb23-43ec-8f90-1c7b39a79967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:44.826430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1347/2688] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:44.834247 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620a9ff-e331-4349-a545-34b5ef19e8c7 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:44.834571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620a9ff-e331-4349-a545-34b5ef19e8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:44.834854 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620a9ff-e331-4349-a545-34b5ef19e8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:44.853240 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7620a9ff-e331-4349-a545-34b5ef19e8c7 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:44.854445 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7620a9ff-e331-4349-a545-34b5ef19e8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:44.863161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1342/2689] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:44.867395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d269c-eb23-43ec-8f90-1c7b39a79967 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:44.878247 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d269c-eb23-43ec-8f90-1c7b39a79967 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:44.882840 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:44.883497 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:44.883812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:44.884092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:44.884817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:39:44.885151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:44.886758 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:44.898197 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1348/2690] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:39:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:39:44.902477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620a9ff-e331-4349-a545-34b5ef19e8c7 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:44.912109 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620a9ff-e331-4349-a545-34b5ef19e8c7 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:44.916755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:44.917409 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:44.917691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:44.917968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:44.918939 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:44.920880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:44.935635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:45.050352 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1343/2691] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:39:44 2022] POST /identity/v3/projects => generated 399 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:45.054416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.064444 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c98f8-6fdc-4c39-a923-444d894e47b4 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.120651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.121348 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.121628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.121902 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.122607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.125193 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:45.214905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1349/2692] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:39:45 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:45.218763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.229293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664d2e1-aa37-4550-8687-4c26d6aa5f7f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.234122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.234778 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.235063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.235416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.236089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.237664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:45.248236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1344/2693] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:45.252997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.264226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6c26c-6bce-4136-9daf-a2824a1e0011 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.269619 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f8500-7ae3-4eb2-9899-12d845a62c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.270296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8500-7ae3-4eb2-9899-12d845a62c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.270581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8500-7ae3-4eb2-9899-12d845a62c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.270862 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8500-7ae3-4eb2-9899-12d845a62c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.271597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f8500-7ae3-4eb2-9899-12d845a62c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.273253 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0f8500-7ae3-4eb2-9899-12d845a62c62 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:45.284470 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1350/2694] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:45.289053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.299483 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22faff5-d2d9-4aa3-90e5-30cda551e81f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.304318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.305224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.305505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.305777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] PATH_INFO: `/v3/projects/aa0a73f196ed476b9f5b49ccc1dfdde0/users/f730fcdb8ab940bd9dfa523fd59f8f22/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.322004 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa0a73f196ed476b9f5b49ccc1dfdde0, user_id=f730fcdb8ab940bd9dfa523fd59f8f22, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.324682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:45.432310 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1345/2695] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:45 2022] PUT /identity/v3/projects/aa0a73f196ed476b9f5b49ccc1dfdde0/users/f730fcdb8ab940bd9dfa523fd59f8f22/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:45.444244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09fbf6-c704-4b85-910c-dc911c045f34 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.447356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09fbf6-c704-4b85-910c-dc911c045f34 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.447356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09fbf6-c704-4b85-910c-dc911c045f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.475263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-fd09fbf6-c704-4b85-910c-dc911c045f34 None None] MFA Rules not processed for user `f730fcdb8ab940bd9dfa523fd59f8f22`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:45.475263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fd09fbf6-c704-4b85-910c-dc911c045f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:45.535484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:45.562751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1351/2696] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:45.569694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.639654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada6740-5486-462f-b486-ba35dcbadb1c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.644342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.644998 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.645281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.645557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.646238 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.648156 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:45.736562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1346/2697] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:39:45 2022] POST /identity/v3/projects => generated 397 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:45.740409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09fbf6-c704-4b85-910c-dc911c045f34 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.751903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09fbf6-c704-4b85-910c-dc911c045f34 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.807299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.808003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.808289 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.808565 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.809590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.811179 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:45.893907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1352/2698] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:39:45 2022] POST /identity/v3/users => generated 305 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:45.897880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.909037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e791e1-2913-4467-8be6-624351d6e03b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.914098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.914754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.915036 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.915444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.916209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.917780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:45.929675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1347/2699] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:45.933786 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:45.943801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74a784-e52f-4ce6-8c26-074ce4f8fd8e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:45.948369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:45.949240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:45.949524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:45.949801 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] PATH_INFO: `/v3/projects/dead44f2daff4885bc15151f6f4c1801/users/78e3be66999a4333a6078de3418c1aa7/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:45.965205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dead44f2daff4885bc15151f6f4c1801, user_id=78e3be66999a4333a6078de3418c1aa7, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:45.968457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:46.031434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1353/2700] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:45 2022] PUT /identity/v3/projects/dead44f2daff4885bc15151f6f4c1801/users/78e3be66999a4333a6078de3418c1aa7/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:46.035313 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:46.045462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5572438-0b95-4e4e-83ff-2a7a74e8841d admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:46.099950 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a730324-5379-4ad1-9cb0-4b83a64cacd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:46.100607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a730324-5379-4ad1-9cb0-4b83a64cacd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:46.100889 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a730324-5379-4ad1-9cb0-4b83a64cacd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:46.101160 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a730324-5379-4ad1-9cb0-4b83a64cacd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:46.101874 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a730324-5379-4ad1-9cb0-4b83a64cacd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:46.103448 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a730324-5379-4ad1-9cb0-4b83a64cacd1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:46.114185 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1348/2701] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:39:46.118822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:46.129441 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ad422-9eb9-4a73-a130-ee01e3472de1 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:46.134080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:46.134948 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:46.135226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:46.135495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] PATH_INFO: `/v3/projects/dead44f2daff4885bc15151f6f4c1801/users/78e3be66999a4333a6078de3418c1aa7/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:46.137590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dead44f2daff4885bc15151f6f4c1801, user_id=78e3be66999a4333a6078de3418c1aa7, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:46.140341 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:46.150534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1354/2702] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:46 2022] PUT /identity/v3/projects/dead44f2daff4885bc15151f6f4c1801/users/78e3be66999a4333a6078de3418c1aa7/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:46.161166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5228152-767a-4fb3-a19a-c45dd2acdf36 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:46.161497 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5228152-767a-4fb3-a19a-c45dd2acdf36 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:46.161783 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5228152-767a-4fb3-a19a-c45dd2acdf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:46.189153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-e5228152-767a-4fb3-a19a-c45dd2acdf36 None None] MFA Rules not processed for user `78e3be66999a4333a6078de3418c1aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:46.190376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e5228152-767a-4fb3-a19a-c45dd2acdf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:46.208251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:46.257761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1349/2703] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:46 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:46.269008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:46.270702 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:46.327960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d8a8d-a173-4134-9801-b24af6630746 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:46.331463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:46.332140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:46.332420 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:46.332694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:46.342003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:46.344187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:46.354014 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1355/2704] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:46 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:46.550828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5228152-767a-4fb3-a19a-c45dd2acdf36 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:46.561309 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5228152-767a-4fb3-a19a-c45dd2acdf36 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:46.564910 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9d0cc-04ff-40c8-ab30-2d7d99f12a01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:46.565570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9d0cc-04ff-40c8-ab30-2d7d99f12a01 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:46.565855 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9d0cc-04ff-40c8-ab30-2d7d99f12a01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:46.566107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9d0cc-04ff-40c8-ab30-2d7d99f12a01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:46.650502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9d0cc-04ff-40c8-ab30-2d7d99f12a01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:46.652116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9d0cc-04ff-40c8-ab30-2d7d99f12a01 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:46.662651 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1350/2705] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:51.830028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:51.883087 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:51.902391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462137c-4e65-4d1a-92c9-afaa751274cb service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:51.906989 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:51.907951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:51.908211 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:51.908536 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] PATH_INFO: `/v3/users/f730fcdb8ab940bd9dfa523fd59f8f22` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:51.909756 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f730fcdb8ab940bd9dfa523fd59f8f22)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:51.911715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:52.073345 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f730fcdb8ab940bd9dfa523fd59f8f22'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:52.085604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f730fcdb8ab940bd9dfa523fd59f8f22'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:52.162199 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f730fcdb8ab940bd9dfa523fd59f8f22'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:52.162834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f730fcdb8ab940bd9dfa523fd59f8f22'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:52.163435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f730fcdb8ab940bd9dfa523fd59f8f22'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:52.176814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1356/2706] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:39:51 2022] DELETE /identity/v3/users/f730fcdb8ab940bd9dfa523fd59f8f22 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:52.181365 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:52.181856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:52.181960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:52.199215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:52.200463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:52.259973 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1351/2707] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:39:52.269597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:52.271041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:52.328303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebb111-e55f-4977-a8b8-31e5556a8a97 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:52.331936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:52.332578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:52.333044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:52.333328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:52.342380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:52.343835 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:52.353388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1357/2708] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:52.780748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:52.781395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:52.794246 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9fe1d-df9f-4054-b63d-21151f9f818c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:52.799613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:52.800559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:52.800938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:52.801121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] PATH_INFO: `/v3/projects/aa0a73f196ed476b9f5b49ccc1dfdde0` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:52.802363 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa0a73f196ed476b9f5b49ccc1dfdde0)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:52.804330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:52.953562 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa0a73f196ed476b9f5b49ccc1dfdde0', 'request_id': 'req-6d404a52-cf3b-45b0-86b3-9057b72b71ca'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:52.955282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:52.961744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa0a73f196ed476b9f5b49ccc1dfdde0', 'request_id': 'req-6d404a52-cf3b-45b0-86b3-9057b72b71ca'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:52.967551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa0a73f196ed476b9f5b49ccc1dfdde0', 'request_id': 'req-6d404a52-cf3b-45b0-86b3-9057b72b71ca'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.042861 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] The token cache is being invalidate because project aa0a73f196ed476b9f5b49ccc1dfdde0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:39:53.043901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.044963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.047661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1352/2709] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:39:52 2022] DELETE /identity/v3/projects/aa0a73f196ed476b9f5b49ccc1dfdde0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:53.052075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:53.062638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e188a-a176-46b6-ad50-4816ccd9d696 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:53.120817 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:53.121695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:53.121981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:53.122260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] PATH_INFO: `/v3/users/78e3be66999a4333a6078de3418c1aa7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:53.123448 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e3be66999a4333a6078de3418c1aa7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:53.125435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:53.130719 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:53.329323 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e3be66999a4333a6078de3418c1aa7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.341830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e3be66999a4333a6078de3418c1aa7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.400986 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e3be66999a4333a6078de3418c1aa7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.401624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e3be66999a4333a6078de3418c1aa7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.402369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-01ed1026-895f-4fbc-836c-164e1eea2172 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e3be66999a4333a6078de3418c1aa7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:53.415344 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1358/2710] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:39:53 2022] DELETE /identity/v3/users/78e3be66999a4333a6078de3418c1aa7 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:53.942870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:53.944764 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:53.959133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d404a52-cf3b-45b0-86b3-9057b72b71ca admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:54.029578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:54.030470 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.030761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.031127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] PATH_INFO: `/v3/projects/dead44f2daff4885bc15151f6f4c1801` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.032431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dead44f2daff4885bc15151f6f4c1801)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:54.034938 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:54.163985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dead44f2daff4885bc15151f6f4c1801', 'request_id': 'req-f88e7982-ca73-4c7c-bca8-636994c02a1f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:54.171186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dead44f2daff4885bc15151f6f4c1801', 'request_id': 'req-f88e7982-ca73-4c7c-bca8-636994c02a1f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:54.177798 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dead44f2daff4885bc15151f6f4c1801', 'request_id': 'req-f88e7982-ca73-4c7c-bca8-636994c02a1f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:54.253393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] The token cache is being invalidate because project dead44f2daff4885bc15151f6f4c1801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:39:54.253710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:54.254348 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-f88e7982-ca73-4c7c-bca8-636994c02a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:54.255966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1353/2711] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:39:53 2022] DELETE /identity/v3/projects/dead44f2daff4885bc15151f6f4c1801 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:54.260879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445900b-b9fe-4e78-bbdd-d14aa1c113dc None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.261376 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445900b-b9fe-4e78-bbdd-d14aa1c113dc None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.261578 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445900b-b9fe-4e78-bbdd-d14aa1c113dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.279465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f445900b-b9fe-4e78-bbdd-d14aa1c113dc None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:54.280830 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f445900b-b9fe-4e78-bbdd-d14aa1c113dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:54.342411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1359/2712] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:54.351237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5b30d-c6aa-48c7-b3db-d5125021aa9a None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.351706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5b30d-c6aa-48c7-b3db-d5125021aa9a None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.352173 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5b30d-c6aa-48c7-b3db-d5125021aa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.371608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-04d5b30d-c6aa-48c7-b3db-d5125021aa9a None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:54.372849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-04d5b30d-c6aa-48c7-b3db-d5125021aa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:54.380756 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1354/2713] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:54.385042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445900b-b9fe-4e78-bbdd-d14aa1c113dc None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:54.397682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445900b-b9fe-4e78-bbdd-d14aa1c113dc None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:54.405355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:54.406160 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.406494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.406846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.407781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:39:54.408201 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:54.410841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:54.428509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1360/2714] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:39:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:39:54.429737 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5b30d-c6aa-48c7-b3db-d5125021aa9a None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:54.441068 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5b30d-c6aa-48c7-b3db-d5125021aa9a None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:54.446793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:54.447726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.448163 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.448563 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.449393 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:54.451538 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:54.562172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1355/2715] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:39:54 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:39:54.563487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:54.575821 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c51c37-edc8-46e5-96c1-10abe5e094ea admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:54.640082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:54.640768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.641049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.641325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.642084 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:54.644714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:54.717050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1361/2716] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:39:54 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:54.722106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:54.732828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503387e-0633-4b99-818d-0bcb4c49fd59 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:54.738106 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:54.744471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.744471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.744471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.744471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:54.744471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:54.754138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1356/2717] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:39:54.758321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:54.763868 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:54.769545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef0cc7e-0b56-4e90-96b3-2f0ee700cd03 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:54.774523 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ac694-8dcf-4520-b3a2-0a1a79fc1339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:54.775195 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ac694-8dcf-4520-b3a2-0a1a79fc1339 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.775479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ac694-8dcf-4520-b3a2-0a1a79fc1339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.775776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ac694-8dcf-4520-b3a2-0a1a79fc1339 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.776537 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ac694-8dcf-4520-b3a2-0a1a79fc1339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:54.778236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ac694-8dcf-4520-b3a2-0a1a79fc1339 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:54.790245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1362/2718] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:54.794849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:54.805994 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c59083-c3bf-445a-8614-d1284fe99d45 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:54.811116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:54.812120 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.812404 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.812766 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] PATH_INFO: `/v3/projects/afd7ccb41f924523befa6facfc0fc814/users/0198820c49a94aa5a4282f66c47ca5b8/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.831458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd7ccb41f924523befa6facfc0fc814, user_id=0198820c49a94aa5a4282f66c47ca5b8, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:54.834162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:54.895024 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1357/2719] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:54 2022] PUT /identity/v3/projects/afd7ccb41f924523befa6facfc0fc814/users/0198820c49a94aa5a4282f66c47ca5b8/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:54.906455 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01cde0-5286-48c1-b2a1-c7436b90c37a None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:54.906883 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01cde0-5286-48c1-b2a1-c7436b90c37a None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:54.907178 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d01cde0-5286-48c1-b2a1-c7436b90c37a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:54.937442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5d01cde0-5286-48c1-b2a1-c7436b90c37a None None] MFA Rules not processed for user `0198820c49a94aa5a4282f66c47ca5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:54.938907 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5d01cde0-5286-48c1-b2a1-c7436b90c37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:55.010728 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1363/2720] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:55.022689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:55.024599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:55.031280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:55.095875 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5e5d6-087c-4016-8768-12a5b7ebdaed admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:55.097690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5595d3c4-ef20-4849-b413-955acdd926f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:55.098353 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d3c4-ef20-4849-b413-955acdd926f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:55.098636 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d3c4-ef20-4849-b413-955acdd926f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:55.098901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5595d3c4-ef20-4849-b413-955acdd926f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:55.108695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595d3c4-ef20-4849-b413-955acdd926f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:55.110164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5595d3c4-ef20-4849-b413-955acdd926f5 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:55.120518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1358/2721] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:39:55.140434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01cde0-5286-48c1-b2a1-c7436b90c37a None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:55.214282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d01cde0-5286-48c1-b2a1-c7436b90c37a None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:55.219060 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:55.220122 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:55.220415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:55.220695 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] PATH_INFO: `/v3/users/0198820c49a94aa5a4282f66c47ca5b8` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:55.222003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0198820c49a94aa5a4282f66c47ca5b8)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:55.224067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:55.402075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0198820c49a94aa5a4282f66c47ca5b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:55.415605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0198820c49a94aa5a4282f66c47ca5b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:55.499682 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0198820c49a94aa5a4282f66c47ca5b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:55.500369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0198820c49a94aa5a4282f66c47ca5b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:55.500977 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0198820c49a94aa5a4282f66c47ca5b8'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:55.512435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:55.517632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1364/2722] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:39:55 2022] DELETE /identity/v3/users/0198820c49a94aa5a4282f66c47ca5b8 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:55.521843 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:55.522140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:55.522421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:55.541701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:55.542930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:55.616475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1359/2723] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:55.627776 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:55.630214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:55.705030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b493a-1071-4264-b9e7-7acf0fc24671 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:55.713419 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927524dd-c780-4e1c-94ba-8387c1fb4073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:55.714123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927524dd-c780-4e1c-94ba-8387c1fb4073 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:55.714410 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927524dd-c780-4e1c-94ba-8387c1fb4073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:55.714690 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927524dd-c780-4e1c-94ba-8387c1fb4073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:55.725803 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927524dd-c780-4e1c-94ba-8387c1fb4073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:55.727181 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927524dd-c780-4e1c-94ba-8387c1fb4073 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:55.737962 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1365/2724] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:39:56.181728 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:56.183303 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:56.196582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68bfbb-fc4a-4f87-8d31-c249e23c9674 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:56.203964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:56.203964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.203964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.203964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] PATH_INFO: `/v3/projects/afd7ccb41f924523befa6facfc0fc814` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.204612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd7ccb41f924523befa6facfc0fc814)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:56.206525 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:56.355432 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd7ccb41f924523befa6facfc0fc814', 'request_id': 'req-65302ac6-f52f-4513-9763-f6f1ae2d5afd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:56.361643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd7ccb41f924523befa6facfc0fc814', 'request_id': 'req-65302ac6-f52f-4513-9763-f6f1ae2d5afd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:56.367171 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd7ccb41f924523befa6facfc0fc814', 'request_id': 'req-65302ac6-f52f-4513-9763-f6f1ae2d5afd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:56.468757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] The token cache is being invalidate because project afd7ccb41f924523befa6facfc0fc814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:39:56.469050 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:56.469683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-65302ac6-f52f-4513-9763-f6f1ae2d5afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:39:56.471212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1360/2725] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:39:56 2022] DELETE /identity/v3/projects/afd7ccb41f924523befa6facfc0fc814 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:56.476383 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e116-2b96-4b4c-b34d-70f25b1d578c None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.476873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e116-2b96-4b4c-b34d-70f25b1d578c None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.476987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e116-2b96-4b4c-b34d-70f25b1d578c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.495242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ae97e116-2b96-4b4c-b34d-70f25b1d578c None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:56.496452 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ae97e116-2b96-4b4c-b34d-70f25b1d578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:56.558901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1366/2726] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:56.567297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804e08-9cfe-47a6-ad6d-a6097f4178ff None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.567580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804e08-9cfe-47a6-ad6d-a6097f4178ff None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.567890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9804e08-9cfe-47a6-ad6d-a6097f4178ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.586601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-c9804e08-9cfe-47a6-ad6d-a6097f4178ff None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:56.587833 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c9804e08-9cfe-47a6-ad6d-a6097f4178ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:56.595608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1361/2727] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:56.599982 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e116-2b96-4b4c-b34d-70f25b1d578c None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:56.609905 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e116-2b96-4b4c-b34d-70f25b1d578c None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:56.614580 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:56.615303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.615582 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.615886 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.616605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:39:56.616910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:56.618512 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:56.630484 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1367/2728] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:39:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:39:56.634840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804e08-9cfe-47a6-ad6d-a6097f4178ff None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:56.644844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9804e08-9cfe-47a6-ad6d-a6097f4178ff None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:56.649584 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:56.650228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.650514 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.650821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.651513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:56.653488 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:56.774344 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1362/2729] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:39:56 2022] POST /identity/v3/projects => generated 377 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:56.778591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:56.788823 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818f3fa-4d8f-4d6b-98f0-3bd3bf92834c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:56.844881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:56.845558 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.845874 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.846507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.847221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:56.849228 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:56.927920 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1368/2730] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:39:56 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:39:56.932214 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:56.937832 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:56.944646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302e89b-9783-44fa-ac8d-beb08e0bdb1b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:56.949506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:56.950182 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.950469 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.950747 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.951539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:56.953132 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:56.966545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1363/2731] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:39:56.970649 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:56.980988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11640f6-e0a3-4ea5-8208-bba9c95d95b3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:56.985716 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccdccf6-6576-4d73-9a6e-4176e40a0a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:56.986382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdccf6-6576-4d73-9a6e-4176e40a0a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:56.987184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdccf6-6576-4d73-9a6e-4176e40a0a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:56.987450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccdccf6-6576-4d73-9a6e-4176e40a0a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:56.988205 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdccf6-6576-4d73-9a6e-4176e40a0a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:56.989820 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccdccf6-6576-4d73-9a6e-4176e40a0a0e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:56.999810 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1369/2732] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:39:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:39:57.006807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:57.017184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f165c-2b27-4b5a-b6af-d124d12afb2b admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:57.022694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:57.023770 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:57.024055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:57.024412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] PATH_INFO: `/v3/projects/e8ae45b29f8c41b59617e9780da7583e/users/5c31ad8b397e4687bd4359e6457d2b9b/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:57.040193 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8ae45b29f8c41b59617e9780da7583e, user_id=5c31ad8b397e4687bd4359e6457d2b9b, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:57.042869 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:57.139847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1364/2733] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:39:57 2022] PUT /identity/v3/projects/e8ae45b29f8c41b59617e9780da7583e/users/5c31ad8b397e4687bd4359e6457d2b9b/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:39:57.151003 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:57.151300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:57.151563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:57.173055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:39:57.185089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] MFA Rules not processed for user `5c31ad8b397e4687bd4359e6457d2b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:39:57.186365 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:57.258227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1370/2734] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:39:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:39:57.269792 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:39:57.272368 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:39:57.354461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9708f45-19b6-4a66-aa35-837cfdbca6a9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:39:57.358039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:39:57.358684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:39:57.358966 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:39:57.359242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:39:57.368328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:39:57.369752 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:39:57.378531 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1365/2735] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:39:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:40:06.936509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:06.936509 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:07.030739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7d026-0ff6-400f-9f9c-bb4197947910 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:07.035908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:07.041652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:07.041652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:07.041652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:07.041652 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:07.054078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:07.055846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:07.066941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1371/2736] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:07.374565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=49,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:17.067577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=69 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:21.694902 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:21.696953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:21.752635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6b84f-5cea-4240-a2cb-ff2891f33a64 service glance] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:21.756232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:21.756873 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:21.757183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:21.757425 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:21.759694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:21.839566 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:21.841020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:21.858746 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1366/2737] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:40:22.785782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:22.786813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:22.876080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e19b90-682d-4ca0-a4e5-7510ee4a0ab3 service placement] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:22.882560 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96931807-91ad-4b03-b4f6-7bc8a12d9b92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:22.883864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931807-91ad-4b03-b4f6-7bc8a12d9b92 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:22.884114 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931807-91ad-4b03-b4f6-7bc8a12d9b92 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:22.884392 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96931807-91ad-4b03-b4f6-7bc8a12d9b92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:22.886173 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-96931807-91ad-4b03-b4f6-7bc8a12d9b92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:22.899034 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96931807-91ad-4b03-b4f6-7bc8a12d9b92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:22.901256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96931807-91ad-4b03-b4f6-7bc8a12d9b92 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:22.914655 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1372/2738] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:26.386421 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:26.447895 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8357d21-68dd-4a4d-8f73-da2893c2961a service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:26.452482 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:26.453376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:26.453657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:26.453924 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] PATH_INFO: `/v3/users/5c31ad8b397e4687bd4359e6457d2b9b` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:26.455097 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c31ad8b397e4687bd4359e6457d2b9b)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:26.457078 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:26.619624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:26.730690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c31ad8b397e4687bd4359e6457d2b9b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:26.747644 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c31ad8b397e4687bd4359e6457d2b9b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:26.834034 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c31ad8b397e4687bd4359e6457d2b9b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:26.834672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c31ad8b397e4687bd4359e6457d2b9b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:26.835301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c31ad8b397e4687bd4359e6457d2b9b'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:26.849060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1367/2739] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:40:26 2022] DELETE /identity/v3/users/5c31ad8b397e4687bd4359e6457d2b9b => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:26.853534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:26.854030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:26.854141 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:26.873812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:26.875026 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:26.937953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1373/2740] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:26.947392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:26.948844 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:27.005903 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4ac9e-671b-4815-b2be-a1da9640c999 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:27.009575 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1bd3a-2c52-4a4a-b01e-646c7dc1384d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:27.010641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1bd3a-2c52-4a4a-b01e-646c7dc1384d service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:27.010641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1bd3a-2c52-4a4a-b01e-646c7dc1384d service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:27.010868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1bd3a-2c52-4a4a-b01e-646c7dc1384d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:27.019884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1bd3a-2c52-4a4a-b01e-646c7dc1384d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:27.021328 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba1bd3a-2c52-4a4a-b01e-646c7dc1384d service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:27.035633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1368/2741] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:27.443288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:27.444804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:27.456822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8862d11-8fba-45d3-a6fc-f534da557a09 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:27.461432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:27.462304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:27.462590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:27.462869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] PATH_INFO: `/v3/projects/e8ae45b29f8c41b59617e9780da7583e` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:27.464095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8ae45b29f8c41b59617e9780da7583e)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:27.466007 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:27.586371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8ae45b29f8c41b59617e9780da7583e', 'request_id': 'req-8ef68816-9d3a-4232-8691-872de3ae5e7d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:27.593325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8ae45b29f8c41b59617e9780da7583e', 'request_id': 'req-8ef68816-9d3a-4232-8691-872de3ae5e7d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:27.599397 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8ae45b29f8c41b59617e9780da7583e', 'request_id': 'req-8ef68816-9d3a-4232-8691-872de3ae5e7d'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:27.675809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] The token cache is being invalidate because project e8ae45b29f8c41b59617e9780da7583e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:40:27.676106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:27.676760 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-8ef68816-9d3a-4232-8691-872de3ae5e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:27.678315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1374/2742] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:40:27 2022] DELETE /identity/v3/projects/e8ae45b29f8c41b59617e9780da7583e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:27.683473 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e131029-76ba-4c9c-be86-4e8bb890d71e None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:27.684011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e131029-76ba-4c9c-be86-4e8bb890d71e None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:27.684125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e131029-76ba-4c9c-be86-4e8bb890d71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:27.703858 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7e131029-76ba-4c9c-be86-4e8bb890d71e None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:27.705073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7e131029-76ba-4c9c-be86-4e8bb890d71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:27.780147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1369/2743] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:27.787918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487563b-62e4-449a-b837-fe7f3b2dc2aa None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:27.788305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487563b-62e4-449a-b837-fe7f3b2dc2aa None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:27.788483 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d487563b-62e4-449a-b837-fe7f3b2dc2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:27.811806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-d487563b-62e4-449a-b837-fe7f3b2dc2aa None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:27.811806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d487563b-62e4-449a-b837-fe7f3b2dc2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:27.818243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1375/2744] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:27.822324 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e131029-76ba-4c9c-be86-4e8bb890d71e None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:27.832323 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e131029-76ba-4c9c-be86-4e8bb890d71e None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:27.837815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:27.838295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:27.838642 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:27.839000 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:27.839814 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:40:27.840128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:27.841683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:27.854670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1370/2745] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:40:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:40:27.860008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487563b-62e4-449a-b837-fe7f3b2dc2aa None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:27.864521 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:27.869881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d487563b-62e4-449a-b837-fe7f3b2dc2aa None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:27.874864 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:27.875520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:27.875841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:27.876121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:27.876824 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:27.878753 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.003314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1376/2746] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:40:27 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:40:28.007535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.018197 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df979fef-b488-438f-8784-441a647ed2c6 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.056918 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:28.093121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.093121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.093505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.093505 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.094282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.096281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.179479 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1371/2747] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:40:28 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:40:28.184251 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.194800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8b2fb-b4c4-4539-8340-525b0fd9c996 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.200281 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.200945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.201227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.201503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.202227 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.203858 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.214040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1377/2748] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:40:28.218223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.228804 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c4bba-68ea-4ceb-b0a6-d5148e10f081 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.233683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd6bb7-4ce8-402f-b006-baea49de1332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.235898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6bb7-4ce8-402f-b006-baea49de1332 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.235898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6bb7-4ce8-402f-b006-baea49de1332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.235898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd6bb7-4ce8-402f-b006-baea49de1332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.235898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6bb7-4ce8-402f-b006-baea49de1332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.237428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd6bb7-4ce8-402f-b006-baea49de1332 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.247975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1372/2749] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:40:28.252081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.262263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d0ff3-9120-4fb7-8fb6-67f0f1e4fd26 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.267031 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.267953 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.268242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.268568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] PATH_INFO: `/v3/projects/cae304adfc1748078f9e1ccfb6e62181/users/4daee202ca9a4ea587209ae493ae0508/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.283304 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cae304adfc1748078f9e1ccfb6e62181, user_id=4daee202ca9a4ea587209ae493ae0508, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.286049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.375057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1378/2750] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:40:28 2022] PUT /identity/v3/projects/cae304adfc1748078f9e1ccfb6e62181/users/4daee202ca9a4ea587209ae493ae0508/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:28.386618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00675b-1ff0-47c4-8ef3-9e9d5e97f2ff None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.386909 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00675b-1ff0-47c4-8ef3-9e9d5e97f2ff None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.387191 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb00675b-1ff0-47c4-8ef3-9e9d5e97f2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.413335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-bb00675b-1ff0-47c4-8ef3-9e9d5e97f2ff None None] MFA Rules not processed for user `4daee202ca9a4ea587209ae493ae0508`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:28.417610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-bb00675b-1ff0-47c4-8ef3-9e9d5e97f2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:28.479159 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1373/2751] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:28.483842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.547628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69951c5-dfbc-4b5b-af10-635f56c394e3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.552234 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.552889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.553169 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.553445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.554123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.556630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.702928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1379/2752] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:40:28 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:40:28.704589 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00675b-1ff0-47c4-8ef3-9e9d5e97f2ff None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.716658 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb00675b-1ff0-47c4-8ef3-9e9d5e97f2ff None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.778647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.779360 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.779641 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.779943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.780640 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.782577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.875249 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1374/2753] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:40:28 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:40:28.879428 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.890502 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1024abf4-2320-4302-ae3c-c4cfd8febadb admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.895361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.896204 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.896528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.896841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.897620 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.899952 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.910416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1380/2754] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:40:28.914361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.924872 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b4ccd1-b4fb-4dba-9a02-10e97ccd4456 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.929522 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804b85c-bc42-44c2-ae2b-0807893deeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.930169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804b85c-bc42-44c2-ae2b-0807893deeac admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.930457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804b85c-bc42-44c2-ae2b-0807893deeac admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.930739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804b85c-bc42-44c2-ae2b-0807893deeac admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.931466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804b85c-bc42-44c2-ae2b-0807893deeac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.933111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804b85c-bc42-44c2-ae2b-0807893deeac admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:28.942940 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1375/2755] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:40:28.947641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:28.957550 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c38428-271b-4e0e-bd62-838e033275c8 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:28.962768 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:28.963638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:28.963955 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:28.964233 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] PATH_INFO: `/v3/projects/3cfb1130cb724605a2ddfb8d31aee88d/users/bf445787580d41eca669d1ce8980a682/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:28.978897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfb1130cb724605a2ddfb8d31aee88d, user_id=bf445787580d41eca669d1ce8980a682, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:28.981585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:29.046856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1381/2756] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:40:28 2022] PUT /identity/v3/projects/3cfb1130cb724605a2ddfb8d31aee88d/users/bf445787580d41eca669d1ce8980a682/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:29.058849 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068ab8b-c705-4f3f-a8b1-b77ad952f3f6 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.059140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068ab8b-c705-4f3f-a8b1-b77ad952f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.059418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068ab8b-c705-4f3f-a8b1-b77ad952f3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.091444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7068ab8b-c705-4f3f-a8b1-b77ad952f3f6 None None] MFA Rules not processed for user `bf445787580d41eca669d1ce8980a682`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:29.091444 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7068ab8b-c705-4f3f-a8b1-b77ad952f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:29.096997 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:29.157914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1376/2757] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:29.162574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:29.221450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:29.227579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6909b33-8cc1-443c-bf88-1d034ed0abf2 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:29.232316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:29.232965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.233245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.233516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.234263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:29.236239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:29.326997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1382/2758] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:40:29 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:40:29.331093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068ab8b-c705-4f3f-a8b1-b77ad952f3f6 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:29.342345 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068ab8b-c705-4f3f-a8b1-b77ad952f3f6 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:29.401153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:29.401831 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.402112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.402386 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.403077 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:29.405037 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:29.497963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1377/2759] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:40:29 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:40:29.502316 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:29.513563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aede96b-3cb1-4def-9018-5a8c7d19696f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:29.518155 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:29.518816 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.519088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.519395 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.520439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:29.522041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:29.533055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1383/2760] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:40:29.537256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:29.547840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88da6a7-c043-41f4-a15f-81c398515075 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:29.552532 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:29.553413 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.553698 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.553978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] PATH_INFO: `/v3/projects/37ba52f1f6b249e1ac72a22c09c401b7/users/5291c44460c949f580b5a5b6cfad5824/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.570053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ba52f1f6b249e1ac72a22c09c401b7, user_id=5291c44460c949f580b5a5b6cfad5824, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:29.573435 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:29.676437 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1378/2761] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:40:29 2022] PUT /identity/v3/projects/37ba52f1f6b249e1ac72a22c09c401b7/users/5291c44460c949f580b5a5b6cfad5824/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:29.681499 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:29.692387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa98176-2324-4426-84d7-93b74f055c6e admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:29.754601 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff52c07-c5f6-4ac5-8000-96c60e5b97c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:29.755286 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52c07-c5f6-4ac5-8000-96c60e5b97c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.755586 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52c07-c5f6-4ac5-8000-96c60e5b97c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.755903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff52c07-c5f6-4ac5-8000-96c60e5b97c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.756639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff52c07-c5f6-4ac5-8000-96c60e5b97c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:29.758865 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff52c07-c5f6-4ac5-8000-96c60e5b97c9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:29.774632 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1384/2762] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:40:29.779080 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:29.789617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff1f063-56c0-46a7-be3b-4683c820de4f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:29.794490 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:29.795389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.795671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.795990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] PATH_INFO: `/v3/projects/37ba52f1f6b249e1ac72a22c09c401b7/users/5291c44460c949f580b5a5b6cfad5824/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.797991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ba52f1f6b249e1ac72a22c09c401b7, user_id=5291c44460c949f580b5a5b6cfad5824, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:29.800659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:29.811573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1379/2763] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:40:29 2022] PUT /identity/v3/projects/37ba52f1f6b249e1ac72a22c09c401b7/users/5291c44460c949f580b5a5b6cfad5824/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:29.822899 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3bde6-16e6-40d0-80bc-112f55fe7cec None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:29.823279 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3bde6-16e6-40d0-80bc-112f55fe7cec None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:29.823584 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3bde6-16e6-40d0-80bc-112f55fe7cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:29.852585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5de3bde6-16e6-40d0-80bc-112f55fe7cec None None] MFA Rules not processed for user `5291c44460c949f580b5a5b6cfad5824`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:29.853927 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5de3bde6-16e6-40d0-80bc-112f55fe7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:29.926136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1385/2764] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:29.938256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:29.939985 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:29.996836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ff1656-9e70-4f65-bbb5-7f24831c7f82 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:30.000529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:30.001175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:30.001461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:30.001740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:30.018602 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:30.058671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:30.060138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:30.069046 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1380/2765] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:33.267898 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3bde6-16e6-40d0-80bc-112f55fe7cec None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:33.278338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3bde6-16e6-40d0-80bc-112f55fe7cec None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:33.282012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:33.282665 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:33.282946 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:33.283221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:33.332850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:33.343098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:33.344589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:33.354897 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1386/2766] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:34.859373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:34.868694 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0532bcfc-f856-44d2-8f24-804ecd922dc7 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:34.872382 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:34.874084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:34.874084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:34.874084 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:34.883388 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:34.884848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:34.894410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1381/2767] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:37.711694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:37.713187 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:37.770426 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df3e17-fd83-4a72-bf7b-87a5dcf18be7 service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:37.774622 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf04f75-f476-4aee-b18a-b0be8a9ba667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:37.776208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf04f75-f476-4aee-b18a-b0be8a9ba667 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:37.776208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf04f75-f476-4aee-b18a-b0be8a9ba667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:37.776208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf04f75-f476-4aee-b18a-b0be8a9ba667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:37.777739 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fbf04f75-f476-4aee-b18a-b0be8a9ba667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:37.844546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf04f75-f476-4aee-b18a-b0be8a9ba667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:37.846047 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf04f75-f476-4aee-b18a-b0be8a9ba667 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:37.856153 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1387/2768] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:39.670573 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:39.685600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fec321-1645-4ebf-a8ad-57a426ac1899 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:39.688613 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:39.689499 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:39.689782 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:39.690060 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] PATH_INFO: `/v3/users/4daee202ca9a4ea587209ae493ae0508` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:39.691264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4daee202ca9a4ea587209ae493ae0508)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:39.693551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:39.878172 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4daee202ca9a4ea587209ae493ae0508'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:39.890347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4daee202ca9a4ea587209ae493ae0508'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:39.942784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4daee202ca9a4ea587209ae493ae0508'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:39.943333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4daee202ca9a4ea587209ae493ae0508'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:39.943990 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4daee202ca9a4ea587209ae493ae0508'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:39.956735 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1382/2769] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:40:39 2022] DELETE /identity/v3/users/4daee202ca9a4ea587209ae493ae0508 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:39.961845 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:39.962432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:39.962432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:39.979909 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:39.981103 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:40.043118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1388/2770] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:40.052649 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:40.054047 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:40.114586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd92f-1e7b-4efc-9a69-ac5d348c9700 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:40.118836 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:40.119489 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:40.119796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:40.120124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:40.129652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:40.131161 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:40.141777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1383/2771] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:40.705714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:40.707997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:40.723855 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be10948-b5a7-45f3-99f6-47ad51e5bffd None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:40.730299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:40.731198 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:40.731489 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:40.731806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] PATH_INFO: `/v3/projects/cae304adfc1748078f9e1ccfb6e62181` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:40.733138 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cae304adfc1748078f9e1ccfb6e62181)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:40.735248 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:40.874241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:40.897465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cae304adfc1748078f9e1ccfb6e62181', 'request_id': 'req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:40.905822 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cae304adfc1748078f9e1ccfb6e62181', 'request_id': 'req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:40.919412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cae304adfc1748078f9e1ccfb6e62181', 'request_id': 'req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.004588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] The token cache is being invalidate because project cae304adfc1748078f9e1ccfb6e62181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:40:41.005175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.005933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.008746 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1389/2772] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:40:40 2022] DELETE /identity/v3/projects/cae304adfc1748078f9e1ccfb6e62181 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:41.013114 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:41.025099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80bf63-d6f7-46a5-b778-c5f594b79ae6 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:41.047846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:41.092720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:41.093610 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:41.094074 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:41.094618 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] PATH_INFO: `/v3/users/bf445787580d41eca669d1ce8980a682` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:41.096523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf445787580d41eca669d1ce8980a682)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:41.099001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:41.354620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf445787580d41eca669d1ce8980a682'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.371577 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf445787580d41eca669d1ce8980a682'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.470245 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf445787580d41eca669d1ce8980a682'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.470980 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf445787580d41eca669d1ce8980a682'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.471695 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf445787580d41eca669d1ce8980a682'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:41.489828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1384/2773] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:40:41 2022] DELETE /identity/v3/users/bf445787580d41eca669d1ce8980a682 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:41.985457 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:41.986949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:42.003873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc99c2-4ec4-4c99-ad37-2563d2fa7486 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:42.077328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:42.078210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:42.078491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:42.078764 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] PATH_INFO: `/v3/projects/3cfb1130cb724605a2ddfb8d31aee88d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:42.080040 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cfb1130cb724605a2ddfb8d31aee88d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:42.082036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:42.203504 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cfb1130cb724605a2ddfb8d31aee88d', 'request_id': 'req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.210015 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cfb1130cb724605a2ddfb8d31aee88d', 'request_id': 'req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.215809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cfb1130cb724605a2ddfb8d31aee88d', 'request_id': 'req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.313819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] The token cache is being invalidate because project 3cfb1130cb724605a2ddfb8d31aee88d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:40:42.314069 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.314675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.316388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1390/2774] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:40:41 2022] DELETE /identity/v3/projects/3cfb1130cb724605a2ddfb8d31aee88d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:42.321086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:42.322629 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:42.335460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d93a05-62b3-46bf-bec5-d47041a54358 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:42.395808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:42.396755 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:42.397082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:42.397395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] PATH_INFO: `/v3/users/5291c44460c949f580b5a5b6cfad5824` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:42.398646 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5291c44460c949f580b5a5b6cfad5824)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:42.400800 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:42.577555 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:42.583600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5291c44460c949f580b5a5b6cfad5824'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.595696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5291c44460c949f580b5a5b6cfad5824'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.692696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5291c44460c949f580b5a5b6cfad5824'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.693318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5291c44460c949f580b5a5b6cfad5824'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.696968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-385d0b12-b9ec-4fa1-9620-12056e097d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5291c44460c949f580b5a5b6cfad5824'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:42.711656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1385/2775] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:40:42 2022] DELETE /identity/v3/users/5291c44460c949f580b5a5b6cfad5824 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:43.191708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:43.193355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:43.207207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539be4c-7f89-4b31-baaf-a4f5b4d953e6 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:43.260658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:43.270642 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:43.271544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:43.271976 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:43.272630 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] PATH_INFO: `/v3/projects/37ba52f1f6b249e1ac72a22c09c401b7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:43.273490 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ba52f1f6b249e1ac72a22c09c401b7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:43.275475 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:43.391020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ba52f1f6b249e1ac72a22c09c401b7', 'request_id': 'req-290a531b-731f-4670-ad54-4afcf8106aff'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:43.398896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ba52f1f6b249e1ac72a22c09c401b7', 'request_id': 'req-290a531b-731f-4670-ad54-4afcf8106aff'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:43.405798 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ba52f1f6b249e1ac72a22c09c401b7', 'request_id': 'req-290a531b-731f-4670-ad54-4afcf8106aff'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:43.475559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] The token cache is being invalidate because project 37ba52f1f6b249e1ac72a22c09c401b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:40:43.475941 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:43.476834 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-290a531b-731f-4670-ad54-4afcf8106aff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:43.478861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1391/2776] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:40:43 2022] DELETE /identity/v3/projects/37ba52f1f6b249e1ac72a22c09c401b7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:43.483616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cabaa3-9204-494c-a668-e24df8adde20 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:43.484722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cabaa3-9204-494c-a668-e24df8adde20 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:43.485178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cabaa3-9204-494c-a668-e24df8adde20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:43.506218 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-55cabaa3-9204-494c-a668-e24df8adde20 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:43.508762 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-55cabaa3-9204-494c-a668-e24df8adde20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:43.576681 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1386/2777] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:40:43.584641 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a13d1-223b-4111-92b4-b17407a20e96 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:43.585025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a13d1-223b-4111-92b4-b17407a20e96 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:43.585262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a13d1-223b-4111-92b4-b17407a20e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:43.605207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-dd5a13d1-223b-4111-92b4-b17407a20e96 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:43.606356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-dd5a13d1-223b-4111-92b4-b17407a20e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:43.622123 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1392/2778] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:43.623021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cabaa3-9204-494c-a668-e24df8adde20 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:43.635439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cabaa3-9204-494c-a668-e24df8adde20 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:43.635439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:43.641638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:43.641638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:43.641638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:43.641638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:40:43.641638 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:43.643261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:43.655988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1387/2779] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:40:43.660698 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a13d1-223b-4111-92b4-b17407a20e96 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:43.670543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a13d1-223b-4111-92b4-b17407a20e96 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:43.675960 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:43.676685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:43.677088 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:43.677430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:43.678208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:43.680891 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:43.800352 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1393/2780] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:40:43 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:40:43.801580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:43.812539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91adc7f5-8fbc-4361-becd-d13ecd47b4a3 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:43.896821 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:43.897597 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:43.897959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:43.898877 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:43.899661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:43.901803 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:43.992280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1388/2781] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:40:43 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:40:43.993511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:44.004329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e537c-e151-4618-b4c3-a5aa471dfd48 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:44.010172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:44.010527 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:44.010808 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:44.011083 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:44.011841 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:44.013518 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:44.024546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1394/2782] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:40:44.028714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:44.039431 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa552e5-0dba-46b4-9ba8-0736de5c08bf admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:44.044418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c147cb3-cc2f-425a-9cc8-1d326176b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:44.045079 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c147cb3-cc2f-425a-9cc8-1d326176b223 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:44.045366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c147cb3-cc2f-425a-9cc8-1d326176b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:44.045685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c147cb3-cc2f-425a-9cc8-1d326176b223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:44.046411 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c147cb3-cc2f-425a-9cc8-1d326176b223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:44.048040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c147cb3-cc2f-425a-9cc8-1d326176b223 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:44.058771 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1389/2783] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:40:44.062896 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:44.074623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af36aeca-04da-4a11-b5b7-d882c10bb8fd admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:44.079500 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:44.080465 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:44.080741 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:44.081013 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] PATH_INFO: `/v3/projects/512bd83d26b94036b9067e2998520c4d/users/f9e999bf4c7e4edda7f004c63a287e8f/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:44.099002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=512bd83d26b94036b9067e2998520c4d, user_id=f9e999bf4c7e4edda7f004c63a287e8f, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:44.101906 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:44.165685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1395/2784] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:40:44 2022] PUT /identity/v3/projects/512bd83d26b94036b9067e2998520c4d/users/f9e999bf4c7e4edda7f004c63a287e8f/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:44.168647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:44.168647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:44.168647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:44.187498 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:44.188704 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:44.209957 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:44.265064 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1390/2785] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:40:44.278964 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:44.281624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:44.372723 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e25c0d-ffdf-448a-9359-5f603e941852 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:44.376496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17445a22-5a9d-4341-bc98-788664c2386e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:44.377194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445a22-5a9d-4341-bc98-788664c2386e service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:44.377476 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445a22-5a9d-4341-bc98-788664c2386e service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:44.377735 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17445a22-5a9d-4341-bc98-788664c2386e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:44.392450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17445a22-5a9d-4341-bc98-788664c2386e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:44.394012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17445a22-5a9d-4341-bc98-788664c2386e service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:44.411115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1396/2786] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:40:45.639516 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:45.641336 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:45.737517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4b764-0754-4f3b-a10e-81de58ff8e80 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:45.743616 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:45.744366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:45.744650 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:45.744925 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:45.746882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:45.761013 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:45.762699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:45.774452 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1391/2787] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:40:45.800300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613041b-6844-4626-9b88-fb745f669894 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:45.800300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613041b-6844-4626-9b88-fb745f669894 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:45.800300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613041b-6844-4626-9b88-fb745f669894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:45.803040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:45.804964 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:45.819276 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642e331-42bd-4145-8629-78f2d7bb6889 service placement] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:45.825170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:45.834229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:45.835586 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:45.836796 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] PATH_INFO: `/v3/users/88575c0bd9db4fa8bc9bd511c0ac37dd` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:45.838910 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-2613041b-6844-4626-9b88-fb745f669894 None None] MFA Rules not processed for user `f9e999bf4c7e4edda7f004c63a287e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:45.839815 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88575c0bd9db4fa8bc9bd511c0ac37dd)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:45.840434 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2613041b-6844-4626-9b88-fb745f669894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:45.843019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:45.852085 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:45.910033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1397/2788] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:45.923002 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613041b-6844-4626-9b88-fb745f669894 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:45.924458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-2613041b-6844-4626-9b88-fb745f669894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:45.986546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613041b-6844-4626-9b88-fb745f669894 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:45.990456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:45.991131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:45.991442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:45.991733 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:46.001789 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:46.003271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:46.015967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1398/2789] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:40:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:40:46.033675 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88575c0bd9db4fa8bc9bd511c0ac37dd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.047492 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88575c0bd9db4fa8bc9bd511c0ac37dd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.144513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88575c0bd9db4fa8bc9bd511c0ac37dd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.148880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88575c0bd9db4fa8bc9bd511c0ac37dd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.148880 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88575c0bd9db4fa8bc9bd511c0ac37dd'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.159542 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1392/2790] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:40:45 2022] DELETE /identity/v3/users/88575c0bd9db4fa8bc9bd511c0ac37dd => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:46.395411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:46.406648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346575-0359-4e50-acaa-e95083975bcd service cinder] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:46.468987 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:46.470004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:46.470278 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:46.470548 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] PATH_INFO: `/v3/projects/87d8cb3f18ae4f05a85dc21abaf316cb` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:46.471729 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87d8cb3f18ae4f05a85dc21abaf316cb)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:46.473657 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:46.590091 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87d8cb3f18ae4f05a85dc21abaf316cb', 'request_id': 'req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.596318 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87d8cb3f18ae4f05a85dc21abaf316cb', 'request_id': 'req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.598412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:46.602416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87d8cb3f18ae4f05a85dc21abaf316cb', 'request_id': 'req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.687331 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] The token cache is being invalidate because project 87d8cb3f18ae4f05a85dc21abaf316cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:40:46.687621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.688325 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:46.689945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1399/2791] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:40:46 2022] DELETE /identity/v3/projects/87d8cb3f18ae4f05a85dc21abaf316cb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:46.694396 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:46.697670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:46.713005 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd66b1-584f-448c-9437-a51a7a28d168 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:46.739423 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:46.780113 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:46.781293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:46.781690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:46.782061 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] PATH_INFO: `/v3/users/0adc392546a54ca5beee09ef53cc6be2` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:46.799145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0adc392546a54ca5beee09ef53cc6be2)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:46.801237 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:47.038009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0adc392546a54ca5beee09ef53cc6be2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.052242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0adc392546a54ca5beee09ef53cc6be2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.119557 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0adc392546a54ca5beee09ef53cc6be2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.120250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0adc392546a54ca5beee09ef53cc6be2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.120846 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-eba141b6-0a36-4853-8ed5-6f751c19cf8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0adc392546a54ca5beee09ef53cc6be2'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.133692 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1393/2792] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:40:46 2022] DELETE /identity/v3/users/0adc392546a54ca5beee09ef53cc6be2 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:47.406596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:40:47.408148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:47.422248 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba91b3f-cb2b-4f10-b1dc-53cdc61f4636 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:40:47.478635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:40:47.479491 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:47.479826 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:47.480131 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] PATH_INFO: `/v3/projects/ddf52d1949a6436da70dc66673708cd6` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:47.490811 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddf52d1949a6436da70dc66673708cd6)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:40:47.493311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:40:47.596065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddf52d1949a6436da70dc66673708cd6', 'request_id': 'req-78500f7b-f942-4f1c-9b4d-2db4cdafb286'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.603267 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddf52d1949a6436da70dc66673708cd6', 'request_id': 'req-78500f7b-f942-4f1c-9b4d-2db4cdafb286'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.609025 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddf52d1949a6436da70dc66673708cd6', 'request_id': 'req-78500f7b-f942-4f1c-9b4d-2db4cdafb286'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.692436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] The token cache is being invalidate because project ddf52d1949a6436da70dc66673708cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:40:47.692706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.693361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-78500f7b-f942-4f1c-9b4d-2db4cdafb286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:40:47.697456 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1400/2793] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:40:47 2022] DELETE /identity/v3/projects/ddf52d1949a6436da70dc66673708cd6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:40:47.779608 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a55b-bdf4-48f4-a888-1ea8e7530061 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:40:47.780270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a55b-bdf4-48f4-a888-1ea8e7530061 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:40:47.780270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b963a55b-bdf4-48f4-a888-1ea8e7530061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:40:47.799601 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-b963a55b-bdf4-48f4-a888-1ea8e7530061 None None] MFA Rules not processed for user `f9e999bf4c7e4edda7f004c63a287e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:40:47.800936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b963a55b-bdf4-48f4-a888-1ea8e7530061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:40:47.871462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1394/2794] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:40:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:40:57.688943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=59,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:40:57.862715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:41:07.139633 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:07.140236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:07.140236 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:07.158431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] MFA Rules not processed for user `f9e999bf4c7e4edda7f004c63a287e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:41:07.159672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:07.167022 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1401/2795] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:41:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:41:17.153396 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:41:32.264142 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e8a9-90a8-4f32-9d00-f3e1c37c5132 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:32.268447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e8a9-90a8-4f32-9d00-f3e1c37c5132 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:32.268447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312e8a9-90a8-4f32-9d00-f3e1c37c5132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:32.325545 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-9312e8a9-90a8-4f32-9d00-f3e1c37c5132 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:41:32.326664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-9312e8a9-90a8-4f32-9d00-f3e1c37c5132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:32.390104 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1395/2796] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:41:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:41:32.519639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:41:32.519639 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:32.612039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02684d-6c73-4d72-af4c-bd300c85dcf6 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:41:32.617161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:41:32.617836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:32.618125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:32.618405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:32.627511 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:41:32.629139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:41:32.637214 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1402/2797] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:41:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:41:42.387075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=44 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:41:42.634967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=41 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:41:49.193693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:49.194417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:49.194417 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:49.212806 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:41:49.214014 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:49.221716 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1396/2798] 10.222.0.42 () {58 vars in 967 bytes} [Thu May 26 21:41:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:41:49.233778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:41:49.244336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7f67a4-7b27-4659-af98-ebd3e9e2a9ac service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:41:49.249284 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:41:49.249913 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:49.250192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:49.250466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:49.259055 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:41:49.260676 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:41:49.269916 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1403/2799] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:41:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 21:41:52.263932 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:41:52.265506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:52.336963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea61672-9753-4fc0-a79c-02a8897117b2 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:41:52.341751 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:41:52.342647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:52.342982 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:52.343209 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] PATH_INFO: `/v3/users/f9e999bf4c7e4edda7f004c63a287e8f` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:52.344422 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9e999bf4c7e4edda7f004c63a287e8f)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:41:52.346893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:41:52.572784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9e999bf4c7e4edda7f004c63a287e8f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:52.585988 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9e999bf4c7e4edda7f004c63a287e8f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:52.667128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9e999bf4c7e4edda7f004c63a287e8f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:52.667774 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9e999bf4c7e4edda7f004c63a287e8f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:52.668372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9e999bf4c7e4edda7f004c63a287e8f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:52.681430 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1397/2800] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:41:52 2022] DELETE /identity/v3/users/f9e999bf4c7e4edda7f004c63a287e8f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:41:52.930917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:41:52.942250 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff923bd1-0dc8-4771-aca9-813e72fbc82b service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:41:53.001673 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:41:53.003048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:53.003335 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:53.003611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] PATH_INFO: `/v3/projects/512bd83d26b94036b9067e2998520c4d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:53.004856 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=512bd83d26b94036b9067e2998520c4d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:41:53.006774 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:41:53.114813 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '512bd83d26b94036b9067e2998520c4d', 'request_id': 'req-aa01253b-b09e-4d29-b8a3-aefccb7831fa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:53.121378 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '512bd83d26b94036b9067e2998520c4d', 'request_id': 'req-aa01253b-b09e-4d29-b8a3-aefccb7831fa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:53.128056 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '512bd83d26b94036b9067e2998520c4d', 'request_id': 'req-aa01253b-b09e-4d29-b8a3-aefccb7831fa'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:53.203161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] The token cache is being invalidate because project 512bd83d26b94036b9067e2998520c4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:41:53.203429 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:53.205051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:41:53.207213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1404/2801] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:41:52 2022] DELETE /identity/v3/projects/512bd83d26b94036b9067e2998520c4d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:41:56.939853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:41:56.941354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:56.997779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abed71fd-3751-411b-b93e-c49d55d4c1c9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:41:57.024647 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:41:57.047186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:41:57.047860 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:57.048145 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:57.048420 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:57.050920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:57.122696 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:41:57.124773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:41:57.136785 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1398/2802] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:41:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:41:58.654722 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:41:58.656271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:58.670347 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01253b-b09e-4d29-b8a3-aefccb7831fa admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:41:58.673951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b8db0-dbbc-442c-a56e-067bc592f43c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:41:58.674597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b8db0-dbbc-442c-a56e-067bc592f43c service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:41:58.674879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b8db0-dbbc-442c-a56e-067bc592f43c service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:41:58.675157 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b8db0-dbbc-442c-a56e-067bc592f43c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:41:58.676949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-b55b8db0-dbbc-442c-a56e-067bc592f43c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:41:58.691531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b8db0-dbbc-442c-a56e-067bc592f43c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:41:58.693028 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b8db0-dbbc-442c-a56e-067bc592f43c service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:41:58.705667 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1405/2803] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:41:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:42:07.131840 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=41 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:42:08.699613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:43:03.662839 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:43:03.672442 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50365f6f-a8f9-4686-bc19-4aedcc0f6d07 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:43:03.676086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:43:03.676729 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:43:03.677011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:43:03.677288 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:43:03.679670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:43:03.689354 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:43:03.690769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:43:03.700592 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1399/2804] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:43:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:43:13.697402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=6 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:44:19.795118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:44:19.795692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:44:19.795692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:44:19.814008 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] MFA Rules not processed for user `63e98f6491344656a94e199575494def`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:44:19.815223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:19.870445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1406/2805] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:44:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:44:20.165993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:44:20.167392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:20.177769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a5de5-1a3c-4ddf-9dd6-f96eb2577d43 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:44:20.182053 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:44:20.182687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:44:20.182992 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:44:20.183261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:44:20.184969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:20.194295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:44:20.195878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:44:20.203965 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1400/2806] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:44:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:44:20.585262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:44:20.591100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:20.663957 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8188241d-707a-4444-a5a3-38cc7ec89d19 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:44:20.669400 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:44:20.670049 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:44:20.670334 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:44:20.670616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:44:20.672418 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:20.684387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:44:20.686024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:44:20.697154 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1407/2807] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:44:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:44:30.200203 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=6 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:44:30.692192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=57 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:44:32.211633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:44:32.214103 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:32.235535 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d142043-dff4-48a8-a6ba-73192be3b8fa service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:44:32.241666 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:44:32.242379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:44:32.242657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:44:32.242927 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:44:32.244968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:32.258851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:44:32.261335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:44:32.273553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1401/2808] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:44:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:44:35.751945 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:44:35.765036 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac634d37-647e-4add-9bb4-fd8b69aafd61 service placement] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:44:35.770481 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:44:35.771126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:44:35.771411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:44:35.771692 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:44:35.773515 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:35.784806 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:44:35.786415 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:44:35.796161 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1408/2809] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:44:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:44:37.135067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:44:37.136518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:37.150436 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c89469-1400-48f4-9964-05bf027d4dc8 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:44:37.157639 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:44:37.158824 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:44:37.159111 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:44:37.159389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:44:37.161677 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:44:37.175035 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:44:37.176657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:44:37.188772 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1402/2810] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:44:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:44:45.791819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:44:47.183807 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=12 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:45:09.928064 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:45:09.939672 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa8bb92-746d-49d5-87ce-b95ab044c9c3 service nova] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:45:09.944645 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:45:09.945300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:45:09.945581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:45:09.945861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:45:09.947629 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:45:09.959487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:45:09.961104 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:45:09.971140 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1409/2811] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:45:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:45:19.966998 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=6 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:10.229777 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:10.241518 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93ce74-d016-4d5c-a973-635832bbb013 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:10.247408 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eefd00457d354271867932d336d7f69c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:10.248098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:10.248625 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:10.248900 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:10.250784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:10.263464 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:10.265070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:10.288044 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1403/2812] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:16.610020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:16.619549 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f204b2-8b15-4935-a1b0-e55e83f5dd2f service placement] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:16.623111 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:16.623779 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:16.624098 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:16.624372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:16.626666 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:16.692039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:16.693520 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:16.705216 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1410/2813] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:20.161526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:20.163298 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:20.179710 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73390bff-66e9-4580-ba47-8ed069cde106 service nova] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:20.190941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:20.190941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:20.190941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:20.190941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:20.192969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:20.207092 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:20.212865 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:20.220162 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1404/2814] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:20.316001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:20.318258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:20.331033 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a93ec-3688-49d5-ae66-7dd9ab053202 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:20.335680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:20.336581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:20.336869 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:20.337150 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] PATH_INFO: `/v3/users/63e98f6491344656a94e199575494def` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:20.338380 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] RBAC: Authorizing `identity:delete_user(user_id=63e98f6491344656a94e199575494def)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:20.340350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:20.547230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63e98f6491344656a94e199575494def'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:20.562382 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63e98f6491344656a94e199575494def'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:20.634183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63e98f6491344656a94e199575494def'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:20.634969 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63e98f6491344656a94e199575494def'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:20.635688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fc6af03c-97be-4111-87ec-cbab7ad9b81b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63e98f6491344656a94e199575494def'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:20.651030 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1411/2815] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:46:20 2022] DELETE /identity/v3/users/63e98f6491344656a94e199575494def => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:20.920225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:20.932595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b4680-566c-40b7-be83-1f3f3fdf297c service placement] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:20.994379 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:20.995250 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:20.995530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:20.995842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] PATH_INFO: `/v3/projects/fe887b2709bc446c8198c3d141c353af` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:20.997112 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe887b2709bc446c8198c3d141c353af)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:20.999002 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:21.112406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe887b2709bc446c8198c3d141c353af', 'request_id': 'req-42892655-ad2d-442c-933c-e337f354d822'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:21.118585 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe887b2709bc446c8198c3d141c353af', 'request_id': 'req-42892655-ad2d-442c-933c-e337f354d822'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:21.124110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe887b2709bc446c8198c3d141c353af', 'request_id': 'req-42892655-ad2d-442c-933c-e337f354d822'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:21.211496 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] The token cache is being invalidate because project fe887b2709bc446c8198c3d141c353af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:46:21.212261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:21.212564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-42892655-ad2d-442c-933c-e337f354d822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:21.214122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1405/2816] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:46:20 2022] DELETE /identity/v3/projects/fe887b2709bc446c8198c3d141c353af => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:21.219139 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf163d4-5dff-47d2-abbc-179372ca3f78 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.219463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf163d4-5dff-47d2-abbc-179372ca3f78 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.219819 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf163d4-5dff-47d2-abbc-179372ca3f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.237311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-9cf163d4-5dff-47d2-abbc-179372ca3f78 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:21.238534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-9cf163d4-5dff-47d2-abbc-179372ca3f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:21.299232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1412/2817] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:46:21.307122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c751-30c7-488a-b16d-749e08764f58 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.307433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c751-30c7-488a-b16d-749e08764f58 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.307701 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3c751-30c7-488a-b16d-749e08764f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.326622 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-adc3c751-30c7-488a-b16d-749e08764f58 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:21.327876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-adc3c751-30c7-488a-b16d-749e08764f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:21.336526 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1406/2818] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:46:21.341052 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf163d4-5dff-47d2-abbc-179372ca3f78 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:21.351024 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf163d4-5dff-47d2-abbc-179372ca3f78 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:21.355607 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:21.356302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.356590 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.356867 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.357594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:46:21.357903 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:21.359514 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:21.361634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:21.372360 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1413/2819] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:46:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:46:21.378088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c751-30c7-488a-b16d-749e08764f58 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:21.387332 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3c751-30c7-488a-b16d-749e08764f58 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:21.392063 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:21.392714 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.393001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.393281 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.393969 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:21.395885 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:21.498726 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1407/2820] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:46:21 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:21.503556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:21.513751 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3b457-674b-4ca8-a97e-74547e54da04 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:21.569757 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:21.570423 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.570707 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.570981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.571675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:21.573624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:21.646573 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1414/2821] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:46:21 2022] POST /identity/v3/users => generated 302 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:21.650530 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:21.661138 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199e473-7fca-4ddd-b402-8678f4c6bc6a admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:21.666093 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:21.666757 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.667045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.667330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.668102 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:21.669682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:21.683110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1408/2822] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:21.687261 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:21.699166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c94cc-7511-4375-a0b4-22fcd9b7b97b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:21.704115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b15c680-bc46-40ba-a908-04a5759e587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:21.704788 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c680-bc46-40ba-a908-04a5759e587e admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.705077 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c680-bc46-40ba-a908-04a5759e587e admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.705538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b15c680-bc46-40ba-a908-04a5759e587e admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.706262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15c680-bc46-40ba-a908-04a5759e587e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:21.708338 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b15c680-bc46-40ba-a908-04a5759e587e admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:21.719669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1415/2823] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:21.724025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:21.734655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87586f23-6e3a-4fc9-b040-2a36657180bb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:21.740254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:21.741198 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.741472 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.741742 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] PATH_INFO: `/v3/projects/ecbf00b5eddb40eaa48d935b459207d7/users/2abbc71f323d409d835f1858ced3d39f/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.754682 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:21.757242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecbf00b5eddb40eaa48d935b459207d7, user_id=2abbc71f323d409d835f1858ced3d39f, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:21.759898 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:21.847565 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1409/2824] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:21 2022] PUT /identity/v3/projects/ecbf00b5eddb40eaa48d935b459207d7/users/2abbc71f323d409d835f1858ced3d39f/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:46:21.860949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103fe9d-cc3a-4a69-b4ee-60a48fd5d76e None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:21.861245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103fe9d-cc3a-4a69-b4ee-60a48fd5d76e None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:21.861522 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103fe9d-cc3a-4a69-b4ee-60a48fd5d76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:21.890207 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f103fe9d-cc3a-4a69-b4ee-60a48fd5d76e None None] MFA Rules not processed for user `2abbc71f323d409d835f1858ced3d39f`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:21.891468 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f103fe9d-cc3a-4a69-b4ee-60a48fd5d76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:21.960781 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1416/2825] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:21.965546 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:22.038150 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97a107b-3ed8-4ef6-97b6-f52c706d0355 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:22.042868 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:22.043529 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.043847 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.044127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.044827 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:22.046750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:22.155559 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1410/2826] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:46:21 2022] POST /identity/v3/projects => generated 405 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:46:22.160302 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103fe9d-cc3a-4a69-b4ee-60a48fd5d76e None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:22.171188 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103fe9d-cc3a-4a69-b4ee-60a48fd5d76e None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:22.230310 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:22.233058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.233058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.233058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.233058 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:22.234569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:22.242551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:22.372129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1417/2827] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:46:22 2022] POST /identity/v3/users => generated 309 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:22.376039 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:22.386314 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2359c61-52eb-4de9-bffc-cd17925d95f2 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:22.390901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:22.392105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.392390 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.392670 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.393447 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:22.395045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:22.406242 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1411/2828] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:22.411215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:22.421915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca688-1ab9-4fcb-a48d-516c1c2c037b admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:22.426623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:22.428677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.428677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.428677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] PATH_INFO: `/v3/projects/c8e855b935474c438ec75d28f1bd7e98/users/ee6cfff649b94f9a8020d9b9bfeef097/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.445295 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e855b935474c438ec75d28f1bd7e98, user_id=ee6cfff649b94f9a8020d9b9bfeef097, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:22.447992 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:22.526191 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1418/2829] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:22 2022] PUT /identity/v3/projects/c8e855b935474c438ec75d28f1bd7e98/users/ee6cfff649b94f9a8020d9b9bfeef097/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:22.530231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:22.540684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87951557-8af2-48ea-a831-b7ba3271b0b1 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:22.598125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0a7a4-a144-4ab2-bcbc-fd7db580188a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:22.598793 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0a7a4-a144-4ab2-bcbc-fd7db580188a admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.599099 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0a7a4-a144-4ab2-bcbc-fd7db580188a admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.599375 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0a7a4-a144-4ab2-bcbc-fd7db580188a admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.600140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0a7a4-a144-4ab2-bcbc-fd7db580188a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:22.601761 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0a7a4-a144-4ab2-bcbc-fd7db580188a admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:22.612098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1412/2830] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:22.616243 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:22.628439 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f12be-fecc-476c-92ed-0d379362ee10 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:22.633041 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:22.633949 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.634230 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.634507 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] PATH_INFO: `/v3/projects/c8e855b935474c438ec75d28f1bd7e98/users/ee6cfff649b94f9a8020d9b9bfeef097/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.636569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e855b935474c438ec75d28f1bd7e98, user_id=ee6cfff649b94f9a8020d9b9bfeef097, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:22.639263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:22.653581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1419/2831] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:22 2022] PUT /identity/v3/projects/c8e855b935474c438ec75d28f1bd7e98/users/ee6cfff649b94f9a8020d9b9bfeef097/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:22.664072 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121eb2-5ad8-49a7-8adc-93b228895275 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.664366 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121eb2-5ad8-49a7-8adc-93b228895275 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.664748 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75121eb2-5ad8-49a7-8adc-93b228895275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.692851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-75121eb2-5ad8-49a7-8adc-93b228895275 None None] MFA Rules not processed for user `ee6cfff649b94f9a8020d9b9bfeef097`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:22.698631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-75121eb2-5ad8-49a7-8adc-93b228895275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:22.752941 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:22.774251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1413/2832] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:22 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:22.785042 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:22.786718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:22.844496 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9905368-60a1-4b1f-ba60-894e5d5fe1d2 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:22.848235 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c21f05-2bcc-4a17-8187-bbea134a2462 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:22.848881 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c21f05-2bcc-4a17-8187-bbea134a2462 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:22.849162 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c21f05-2bcc-4a17-8187-bbea134a2462 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:22.849442 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c21f05-2bcc-4a17-8187-bbea134a2462 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:22.858613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c21f05-2bcc-4a17-8187-bbea134a2462 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:22.860605 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c21f05-2bcc-4a17-8187-bbea134a2462 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:22.870089 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1420/2833] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:22 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:22.997788 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75121eb2-5ad8-49a7-8adc-93b228895275 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:23.066020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75121eb2-5ad8-49a7-8adc-93b228895275 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:23.070667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:23.071591 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:23.071901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:23.072178 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] PATH_INFO: `/v3/users/2abbc71f323d409d835f1858ced3d39f` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:23.073412 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=2abbc71f323d409d835f1858ced3d39f)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:23.075340 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:23.287222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2abbc71f323d409d835f1858ced3d39f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:23.301031 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2abbc71f323d409d835f1858ced3d39f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:23.416307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2abbc71f323d409d835f1858ced3d39f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:23.416968 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2abbc71f323d409d835f1858ced3d39f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:23.417534 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2abbc71f323d409d835f1858ced3d39f'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:23.431502 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1414/2834] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:46:22 2022] DELETE /identity/v3/users/2abbc71f323d409d835f1858ced3d39f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:23.436159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:23.436664 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:23.436771 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:23.454377 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:23.465240 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:23.513807 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1421/2835] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:23.523477 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:23.525082 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:23.582457 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c51274-62d0-4c81-942a-707b848b31ba admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:23.585952 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:23.586664 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:23.586943 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:23.587219 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:23.597229 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:23.598652 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:23.608199 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1415/2836] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:24.045239 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:24.046677 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:24.059812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d625e21-6874-45ed-b533-5604db60b801 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:24.064689 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:24.066232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:24.066588 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:24.066965 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] PATH_INFO: `/v3/projects/ecbf00b5eddb40eaa48d935b459207d7` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:24.068314 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecbf00b5eddb40eaa48d935b459207d7)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:24.070275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:24.190421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecbf00b5eddb40eaa48d935b459207d7', 'request_id': 'req-827b30fe-89f9-4777-b08b-16a3d62fb1d7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.190421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:24.193783 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecbf00b5eddb40eaa48d935b459207d7', 'request_id': 'req-827b30fe-89f9-4777-b08b-16a3d62fb1d7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.199436 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecbf00b5eddb40eaa48d935b459207d7', 'request_id': 'req-827b30fe-89f9-4777-b08b-16a3d62fb1d7'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.266164 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] The token cache is being invalidate because project ecbf00b5eddb40eaa48d935b459207d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:46:24.266445 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.267148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.269137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1422/2837] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:46:24 2022] DELETE /identity/v3/projects/ecbf00b5eddb40eaa48d935b459207d7 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:24.273633 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:24.284595 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d8bf3f-9bc3-45ad-9a63-cc7e0dbe9790 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:24.290329 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:24.349654 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:24.350753 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:24.351407 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:24.351946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] PATH_INFO: `/v3/users/ee6cfff649b94f9a8020d9b9bfeef097` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:24.353784 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee6cfff649b94f9a8020d9b9bfeef097)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:24.355870 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:24.562763 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee6cfff649b94f9a8020d9b9bfeef097'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.575239 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee6cfff649b94f9a8020d9b9bfeef097'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.661009 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee6cfff649b94f9a8020d9b9bfeef097'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.661659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee6cfff649b94f9a8020d9b9bfeef097'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.662256 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-3f08ee49-49ee-427f-9638-207be45df43d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee6cfff649b94f9a8020d9b9bfeef097'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:24.675871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1416/2838] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:46:24 2022] DELETE /identity/v3/users/ee6cfff649b94f9a8020d9b9bfeef097 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:25.117409 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:25.117894 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:25.132048 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827b30fe-89f9-4777-b08b-16a3d62fb1d7 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:25.192708 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:25.194935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.194935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.194935 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] PATH_INFO: `/v3/projects/c8e855b935474c438ec75d28f1bd7e98` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.195355 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e855b935474c438ec75d28f1bd7e98)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:25.197315 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:25.304300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e855b935474c438ec75d28f1bd7e98', 'request_id': 'req-a7c7f05e-7e04-4f88-af12-65bf4daa559f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:25.310754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e855b935474c438ec75d28f1bd7e98', 'request_id': 'req-a7c7f05e-7e04-4f88-af12-65bf4daa559f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:25.317051 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e855b935474c438ec75d28f1bd7e98', 'request_id': 'req-a7c7f05e-7e04-4f88-af12-65bf4daa559f'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:25.399901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] The token cache is being invalidate because project c8e855b935474c438ec75d28f1bd7e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:46:25.399901 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:25.400547 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a7c7f05e-7e04-4f88-af12-65bf4daa559f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:25.402148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1423/2839] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:46:25 2022] DELETE /identity/v3/projects/c8e855b935474c438ec75d28f1bd7e98 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:25.407959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f900c-0866-4a70-8122-47da38362ed3 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.408261 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f900c-0866-4a70-8122-47da38362ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.408544 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f900c-0866-4a70-8122-47da38362ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.426960 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-fc8f900c-0866-4a70-8122-47da38362ed3 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:25.428225 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-fc8f900c-0866-4a70-8122-47da38362ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:25.489110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1417/2840] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:46:25.496833 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a25cd-c660-42ae-a044-3a200bacd6aa None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.497118 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a25cd-c660-42ae-a044-3a200bacd6aa None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.497398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a25cd-c660-42ae-a044-3a200bacd6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.516430 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-db7a25cd-c660-42ae-a044-3a200bacd6aa None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:25.517685 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-db7a25cd-c660-42ae-a044-3a200bacd6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:25.525928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1424/2841] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:25.530264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f900c-0866-4a70-8122-47da38362ed3 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:25.541118 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f900c-0866-4a70-8122-47da38362ed3 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:25.546128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:25.546801 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.547110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.547389 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.548754 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:46:25.549059 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:25.550655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:25.562933 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1418/2842] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:46:25.567357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a25cd-c660-42ae-a044-3a200bacd6aa None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:25.576900 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a25cd-c660-42ae-a044-3a200bacd6aa None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:25.581720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:25.582391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.583294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.583294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.583669 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:25.585615 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:25.682075 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1425/2843] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:46:25 2022] POST /identity/v3/projects => generated 355 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:25.686331 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:25.696543 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735cab54-8341-40a0-8704-41860df2fb89 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:25.709579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:25.763631 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:25.764335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.764617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.764891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.765582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:25.767480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:25.846342 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1419/2844] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:46:25 2022] POST /identity/v3/users => generated 278 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:25.850505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:25.861320 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c656b893-f43c-40dc-93ab-fa342835cbd3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:25.865973 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:25.866628 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.866908 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.867184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.867938 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:25.869546 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:25.880656 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1426/2845] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:46:25.884624 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:25.894820 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852264b4-2838-4c88-b3cd-b8f32fd1bd69 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:25.899523 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95aaae-69e9-4bb4-82d7-a7b38cc6eae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:25.900220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95aaae-69e9-4bb4-82d7-a7b38cc6eae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.900510 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95aaae-69e9-4bb4-82d7-a7b38cc6eae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.900835 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95aaae-69e9-4bb4-82d7-a7b38cc6eae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.901517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95aaae-69e9-4bb4-82d7-a7b38cc6eae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:25.903110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95aaae-69e9-4bb4-82d7-a7b38cc6eae1 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:25.913433 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1420/2846] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:25.917568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:25.927997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc60604-8c4a-4155-ae7c-55f70eda796c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:25.932623 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:25.933438 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:25.933717 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:25.933991 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] PATH_INFO: `/v3/projects/542f257c120e46609c5f6f3537777d87/users/b445d23cf5424ff983e55c629edeb949/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:25.949063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] RBAC: Authorizing `identity:create_grant(project_id=542f257c120e46609c5f6f3537777d87, user_id=b445d23cf5424ff983e55c629edeb949, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:25.951704 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:26.057021 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1427/2847] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:25 2022] PUT /identity/v3/projects/542f257c120e46609c5f6f3537777d87/users/b445d23cf5424ff983e55c629edeb949/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:26.069007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb70d-05eb-4998-89cc-9ec26bfa8c1b None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.069301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb70d-05eb-4998-89cc-9ec26bfa8c1b None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.069582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dddb70d-05eb-4998-89cc-9ec26bfa8c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.095725 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-2dddb70d-05eb-4998-89cc-9ec26bfa8c1b None None] MFA Rules not processed for user `b445d23cf5424ff983e55c629edeb949`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:26.096959 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2dddb70d-05eb-4998-89cc-9ec26bfa8c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:26.161907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1421/2848] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:26 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:26.166648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:26.176458 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:26.229934 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a2b563-9a78-4ff0-bdf3-c05ab218222c admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:26.234747 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:26.236106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.236388 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.236675 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.237805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:26.240299 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:26.353363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1428/2849] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:46:26 2022] POST /identity/v3/projects => generated 355 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:26.357184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb70d-05eb-4998-89cc-9ec26bfa8c1b None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:26.368116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dddb70d-05eb-4998-89cc-9ec26bfa8c1b None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:26.424062 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:26.424745 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.425030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.425307 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.426011 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:26.427975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:26.535893 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1422/2850] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:46:26 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:26.539860 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:26.550800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957840e9-dbab-4157-bc49-de4199b175f0 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:26.555381 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:26.557078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.557078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.557078 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.557372 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:26.558943 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:26.570136 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1429/2851] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:26.574439 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:26.585169 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886ab00f-59e6-461f-a519-8856d28f13c4 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:26.589811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:26.590702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.590999 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.591260 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] PATH_INFO: `/v3/projects/9df74d4cf01a4a6e84cdc3e47c3d1115/users/de477473f1a84280b8f7c052dd336f65/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.602811 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:26.606791 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df74d4cf01a4a6e84cdc3e47c3d1115, user_id=de477473f1a84280b8f7c052dd336f65, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:26.609480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:26.732255 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1423/2852] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:26 2022] PUT /identity/v3/projects/9df74d4cf01a4a6e84cdc3e47c3d1115/users/de477473f1a84280b8f7c052dd336f65/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:26.736263 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:26.747393 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5597848-ee38-4c3f-9047-4c49f18aeaca admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:26.807700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fd5eb-a993-49a9-baf0-b02890d32e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:26.808322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd5eb-a993-49a9-baf0-b02890d32e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.808611 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd5eb-a993-49a9-baf0-b02890d32e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.808887 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd5eb-a993-49a9-baf0-b02890d32e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.809610 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fd5eb-a993-49a9-baf0-b02890d32e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:26.811213 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fd5eb-a993-49a9-baf0-b02890d32e40 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:26.821660 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1430/2853] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:26.826096 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:26.837387 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadf189-79be-4b31-9666-eb1ef8996005 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:26.842205 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:26.843098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.843385 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.843661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] PATH_INFO: `/v3/projects/9df74d4cf01a4a6e84cdc3e47c3d1115/users/de477473f1a84280b8f7c052dd336f65/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.846293 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df74d4cf01a4a6e84cdc3e47c3d1115, user_id=de477473f1a84280b8f7c052dd336f65, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:26.848995 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:26.860780 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1424/2854] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:26 2022] PUT /identity/v3/projects/9df74d4cf01a4a6e84cdc3e47c3d1115/users/de477473f1a84280b8f7c052dd336f65/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:26.871175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6a0e4-1ac1-4411-af56-16e255a235a3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:26.871463 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6a0e4-1ac1-4411-af56-16e255a235a3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:26.871765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb6a0e4-1ac1-4411-af56-16e255a235a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:26.900710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5cb6a0e4-1ac1-4411-af56-16e255a235a3 None None] MFA Rules not processed for user `de477473f1a84280b8f7c052dd336f65`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:26.902095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5cb6a0e4-1ac1-4411-af56-16e255a235a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:26.976106 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1431/2855] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:26 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:26.987871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:26.989537 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:27.046480 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92d8730-2f5d-4baf-aafc-a152a317f9bb admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:27.050690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:27.051326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:27.051605 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:27.051914 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:27.060913 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:27.062264 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:27.071599 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1425/2856] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:26 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:27.267981 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6a0e4-1ac1-4411-af56-16e255a235a3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:27.272929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:27.278861 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb6a0e4-1ac1-4411-af56-16e255a235a3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:27.283933 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cc4381-1b79-43a5-b57b-f5842e20a145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:27.284594 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4381-1b79-43a5-b57b-f5842e20a145 service cinder] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:27.284914 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4381-1b79-43a5-b57b-f5842e20a145 service cinder] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:27.285215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cc4381-1b79-43a5-b57b-f5842e20a145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:27.343597 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4381-1b79-43a5-b57b-f5842e20a145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:27.345595 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cc4381-1b79-43a5-b57b-f5842e20a145 service cinder] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:27.354459 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1432/2857] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:27 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:37.067512 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=35,keystone:INSERT=2 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:37.386145 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=27 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:44.840235 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:44.906089 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9556ce-20c4-483f-ad92-d780de2ff956 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:44.911234 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:44.912391 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:44.912671 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:44.912946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] PATH_INFO: `/v3/users/b445d23cf5424ff983e55c629edeb949` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:44.914116 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b445d23cf5424ff983e55c629edeb949)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:44.916095 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:45.097280 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b445d23cf5424ff983e55c629edeb949'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.109048 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b445d23cf5424ff983e55c629edeb949'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.193580 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b445d23cf5424ff983e55c629edeb949'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.194221 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b445d23cf5424ff983e55c629edeb949'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.194828 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b445d23cf5424ff983e55c629edeb949'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.209684 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1426/2858] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:46:44 2022] DELETE /identity/v3/users/b445d23cf5424ff983e55c629edeb949 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:45.215836 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:45.216416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:45.216416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:45.234559 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:45.235815 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:45.297769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1433/2859] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 21:46:45.307583 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:45.309054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:45.364842 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5f3288-c038-4a89-9e79-811b05026d40 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:45.368413 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:45.369054 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:45.369333 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:45.369606 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:45.378461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:45.379894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:45.389088 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1427/2860] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:45.792237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:45.792679 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:45.805269 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91253680-6f30-4df5-9275-4ab98a22d864 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:45.810095 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:45.811039 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:45.812221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:45.812543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] PATH_INFO: `/v3/projects/542f257c120e46609c5f6f3537777d87` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:45.814125 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=542f257c120e46609c5f6f3537777d87)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:45.816718 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:45.925581 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542f257c120e46609c5f6f3537777d87', 'request_id': 'req-ae337da5-4a97-4040-a818-6eb2a2f503d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.931680 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542f257c120e46609c5f6f3537777d87', 'request_id': 'req-ae337da5-4a97-4040-a818-6eb2a2f503d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.937232 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542f257c120e46609c5f6f3537777d87', 'request_id': 'req-ae337da5-4a97-4040-a818-6eb2a2f503d9'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.992937 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] The token cache is being invalidate because project 542f257c120e46609c5f6f3537777d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:46:45.993221 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.993837 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:45.995411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1434/2861] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:46:45 2022] DELETE /identity/v3/projects/542f257c120e46609c5f6f3537777d87 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:45.999361 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:46.009655 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820d806-5683-4336-9c70-b0c1f9c96649 service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:46.028295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:46.066620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:46.067715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:46.068224 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:46.068857 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] PATH_INFO: `/v3/users/de477473f1a84280b8f7c052dd336f65` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:46.070139 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=de477473f1a84280b8f7c052dd336f65)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:46.072115 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:46.266023 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de477473f1a84280b8f7c052dd336f65'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:46.283070 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de477473f1a84280b8f7c052dd336f65'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:46.380065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de477473f1a84280b8f7c052dd336f65'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:46.380715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de477473f1a84280b8f7c052dd336f65'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:46.381318 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-df629c50-5352-4285-805b-2a7b768450ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de477473f1a84280b8f7c052dd336f65'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:46.394438 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1428/2862] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:46:45 2022] DELETE /identity/v3/users/de477473f1a84280b8f7c052dd336f65 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:46.790812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:46.800018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:46.806694 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae337da5-4a97-4040-a818-6eb2a2f503d9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:46.870144 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:46.871012 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:46.871292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:46.871568 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] PATH_INFO: `/v3/projects/9df74d4cf01a4a6e84cdc3e47c3d1115` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:46.872812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df74d4cf01a4a6e84cdc3e47c3d1115)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:46.874737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:46.967018 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:46.984487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df74d4cf01a4a6e84cdc3e47c3d1115', 'request_id': 'req-f840f124-a505-400e-91af-d106707dcd9a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:46.991249 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df74d4cf01a4a6e84cdc3e47c3d1115', 'request_id': 'req-f840f124-a505-400e-91af-d106707dcd9a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:46.996928 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df74d4cf01a4a6e84cdc3e47c3d1115', 'request_id': 'req-f840f124-a505-400e-91af-d106707dcd9a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:47.096184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] The token cache is being invalidate because project 9df74d4cf01a4a6e84cdc3e47c3d1115 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:46:47.096421 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:47.097001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-f840f124-a505-400e-91af-d106707dcd9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:47.098524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1435/2863] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:46:46 2022] DELETE /identity/v3/projects/9df74d4cf01a4a6e84cdc3e47c3d1115 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:47.103645 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85db09f-f52f-471e-a4c9-4228af83226d None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.106133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85db09f-f52f-471e-a4c9-4228af83226d None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.106133 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85db09f-f52f-471e-a4c9-4228af83226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.122494 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-a85db09f-f52f-471e-a4c9-4228af83226d None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:47.123817 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-a85db09f-f52f-471e-a4c9-4228af83226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:47.184019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1429/2864] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:47.193275 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c6ac-6d80-4049-aa79-883dc9f332c1 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.193566 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c6ac-6d80-4049-aa79-883dc9f332c1 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.193846 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c6ac-6d80-4049-aa79-883dc9f332c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.212389 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-54c4c6ac-6d80-4049-aa79-883dc9f332c1 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:47.213613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-54c4c6ac-6d80-4049-aa79-883dc9f332c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:47.221495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1436/2865] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:47.225717 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85db09f-f52f-471e-a4c9-4228af83226d None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:47.235402 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85db09f-f52f-471e-a4c9-4228af83226d None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:47.240020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:47.240672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.240951 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.241222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.241949 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80757) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:46:47.242257 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:47.243856 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:47.255911 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1430/2866] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:46:47.260067 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c6ac-6d80-4049-aa79-883dc9f332c1 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:47.269883 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c6ac-6d80-4049-aa79-883dc9f332c1 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:47.274648 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:47.275294 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.275577 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.275890 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.276574 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:47.278487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:47.372654 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1437/2867] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:46:47 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:47.376683 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:47.387175 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa755e-e13b-4d03-957b-1e4317afe6ff admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:47.413657 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:47.444392 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:47.445067 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.445347 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.445621 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.446358 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:47.448330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:47.555231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1431/2868] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:46:47 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:47.559720 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:47.572486 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ab3e61-e3d2-49af-99ae-0ad422cd4b01 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:47.577330 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:47.578010 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.578296 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.578576 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.579356 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:47.581506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:47.592915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1438/2869] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:47.597864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:47.608295 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3df78-3bb9-4e98-b52c-e032157c2afa admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:47.613073 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf5068-0044-43c1-91f1-c678f2ef5c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:47.613739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5068-0044-43c1-91f1-c678f2ef5c77 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.614028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5068-0044-43c1-91f1-c678f2ef5c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.614299 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5068-0044-43c1-91f1-c678f2ef5c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.615012 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf5068-0044-43c1-91f1-c678f2ef5c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:47.616635 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf5068-0044-43c1-91f1-c678f2ef5c77 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:47.627350 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1432/2870] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:47.631427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:47.641329 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ccd9f-dda9-4648-98b6-c7f9dd720269 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:47.646431 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:47.647303 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.647589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.647893 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] PATH_INFO: `/v3/projects/dd79164d103e48c3b9694c7853fefc2d/users/cf0596af7f5142809af09bb080be205b/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.662929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd79164d103e48c3b9694c7853fefc2d, user_id=cf0596af7f5142809af09bb080be205b, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:47.665604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:47.765288 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1439/2871] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:47 2022] PUT /identity/v3/projects/dd79164d103e48c3b9694c7853fefc2d/users/cf0596af7f5142809af09bb080be205b/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:47.781547 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e84d97-b4b7-4a40-a057-964ad2c11d7c None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.781935 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e84d97-b4b7-4a40-a057-964ad2c11d7c None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.782223 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e84d97-b4b7-4a40-a057-964ad2c11d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.808993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-99e84d97-b4b7-4a40-a057-964ad2c11d7c None None] MFA Rules not processed for user `cf0596af7f5142809af09bb080be205b`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:47.810184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-99e84d97-b4b7-4a40-a057-964ad2c11d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:47.875122 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1433/2872] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:47.880530 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:47.941703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790993c-e52b-4701-b849-3065c9449f2f admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:47.946364 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:47.947020 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:47.947306 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:47.947587 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:47.948321 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:47.950224 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:48.052967 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1440/2873] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:46:47 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:48.057507 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e84d97-b4b7-4a40-a057-964ad2c11d7c None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:48.068823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e84d97-b4b7-4a40-a057-964ad2c11d7c None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:48.133779 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:48.134656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:48.134656 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:48.134891 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] PATH_INFO: `/v3/users` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:48.135578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:48.137539 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:48.217121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1434/2874] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:46:48 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:48.221194 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:48.231508 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868d5aaf-f9f6-4fe1-b0dd-254d24b0e5b9 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:48.236149 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:48.236850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:48.237134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:48.237412 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:48.238156 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:48.239852 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:48.250461 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1441/2875] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:48.255287 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:48.266168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb19ef-cb63-4dd0-8ec6-135e65cefac7 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:48.271620 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:48.274254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:48.274254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:48.274254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] PATH_INFO: `/v3/projects/9ec071fe5a7c47e89b6aef73529f9bc6/users/cdb6f42b876b40c685b07b5345bcfc4a/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:48.289164 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec071fe5a7c47e89b6aef73529f9bc6, user_id=cdb6f42b876b40c685b07b5345bcfc4a, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:48.291829 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:48.360025 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1435/2876] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:48 2022] PUT /identity/v3/projects/9ec071fe5a7c47e89b6aef73529f9bc6/users/cdb6f42b876b40c685b07b5345bcfc4a/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:48.370814 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:48.382200 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02950862-5dd5-4682-9904-2fe6344ded49 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:48.402917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:48.442624 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75acfc7-3ca6-4224-b408-80b4da59e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:48.443353 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75acfc7-3ca6-4224-b408-80b4da59e40b admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:48.443635 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75acfc7-3ca6-4224-b408-80b4da59e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:48.444009 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75acfc7-3ca6-4224-b408-80b4da59e40b admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:48.444732 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75acfc7-3ca6-4224-b408-80b4da59e40b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:48.446326 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75acfc7-3ca6-4224-b408-80b4da59e40b admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:48.456895 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1442/2877] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:48 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:48.461222 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:48.467167 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:48.472467 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6075ee2-016b-49eb-93c2-f54153945aa0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:48.476989 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:48.477892 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:48.478184 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:48.478462 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] PATH_INFO: `/v3/projects/9ec071fe5a7c47e89b6aef73529f9bc6/users/cdb6f42b876b40c685b07b5345bcfc4a/roles/0515c8db0fd849e5a6fdd531600c97db` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:48.480461 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec071fe5a7c47e89b6aef73529f9bc6, user_id=cdb6f42b876b40c685b07b5345bcfc4a, role_id=0515c8db0fd849e5a6fdd531600c97db)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:48.483040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:48.493689 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1436/2878] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:48 2022] PUT /identity/v3/projects/9ec071fe5a7c47e89b6aef73529f9bc6/users/cdb6f42b876b40c685b07b5345bcfc4a/roles/0515c8db0fd849e5a6fdd531600c97db => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:48.504944 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9a8ab-99ed-48a1-8fe4-3cbc9a3533c3 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:48.505237 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9a8ab-99ed-48a1-8fe4-3cbc9a3533c3 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:48.505519 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d9a8ab-99ed-48a1-8fe4-3cbc9a3533c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:48.531556 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f7d9a8ab-99ed-48a1-8fe4-3cbc9a3533c3 None None] MFA Rules not processed for user `cdb6f42b876b40c685b07b5345bcfc4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:48.532805 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f7d9a8ab-99ed-48a1-8fe4-3cbc9a3533c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:48.603996 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1443/2879] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:48.615680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:48.617419 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:48.680864 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff169e-212c-4fab-b953-55d427bc568f admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:48.684612 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d56a2b-969f-4fda-ac5c-f0f7ff28f64f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:48.685269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56a2b-969f-4fda-ac5c-f0f7ff28f64f service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:48.685553 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56a2b-969f-4fda-ac5c-f0f7ff28f64f service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:48.685838 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d56a2b-969f-4fda-ac5c-f0f7ff28f64f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:48.695418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d56a2b-969f-4fda-ac5c-f0f7ff28f64f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:48.698157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d56a2b-969f-4fda-ac5c-f0f7ff28f64f service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:48.707688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1437/2880] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 21:46:49.239258 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9a8ab-99ed-48a1-8fe4-3cbc9a3533c3 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:49.307301 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d9a8ab-99ed-48a1-8fe4-3cbc9a3533c3 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:49.312001 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:49.312892 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:49.313166 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:49.313435 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] PATH_INFO: `/v3/users/cf0596af7f5142809af09bb080be205b` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:49.314681 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf0596af7f5142809af09bb080be205b)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:49.316659 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:49.503037 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf0596af7f5142809af09bb080be205b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:49.504168 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:49.515256 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf0596af7f5142809af09bb080be205b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:49.582551 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf0596af7f5142809af09bb080be205b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:49.583183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf0596af7f5142809af09bb080be205b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:49.583812 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf0596af7f5142809af09bb080be205b'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:49.597391 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1444/2881] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:46:49 2022] DELETE /identity/v3/users/cf0596af7f5142809af09bb080be205b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:49.602486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:49.607297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:49.607297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:49.622297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:49.624110 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:49.689121 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1438/2882] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:49.699592 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:49.701516 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:49.772215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683b952-0f7e-43cd-9d98-c6012fdade31 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:49.775829 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:49.776479 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:49.776765 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:49.777044 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:49.786300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:49.788023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:49.797451 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1445/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:46:50.267416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:50.268978 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:50.284773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9fa72b-a17b-46f7-987b-003c5b482777 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:50.290520 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:50.291410 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:50.291688 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:50.292001 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] PATH_INFO: `/v3/projects/dd79164d103e48c3b9694c7853fefc2d` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:50.293416 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd79164d103e48c3b9694c7853fefc2d)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:50.295357 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:50.403598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd79164d103e48c3b9694c7853fefc2d', 'request_id': 'req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.411016 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd79164d103e48c3b9694c7853fefc2d', 'request_id': 'req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.417107 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd79164d103e48c3b9694c7853fefc2d', 'request_id': 'req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.500364 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] The token cache is being invalidate because project dd79164d103e48c3b9694c7853fefc2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:46:50.502459 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.505148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.509963 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1439/2884] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:46:50 2022] DELETE /identity/v3/projects/dd79164d103e48c3b9694c7853fefc2d => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:50.510109 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:50.521133 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacb745-05e5-4e0e-87eb-8bb47d4f895b service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:50.579524 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:50.580387 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:50.580700 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:50.581478 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] PATH_INFO: `/v3/users/cdb6f42b876b40c685b07b5345bcfc4a` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:50.582754 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdb6f42b876b40c685b07b5345bcfc4a)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:50.584743 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:50.762842 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdb6f42b876b40c685b07b5345bcfc4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.775606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdb6f42b876b40c685b07b5345bcfc4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.779336 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:50.879769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdb6f42b876b40c685b07b5345bcfc4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.880322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdb6f42b876b40c685b07b5345bcfc4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.880918 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-fda28cd6-3537-4926-a367-a67f4e370c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdb6f42b876b40c685b07b5345bcfc4a'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:50.894404 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1446/2885] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:46:50 2022] DELETE /identity/v3/users/cdb6f42b876b40c685b07b5345bcfc4a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:51.309511 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:51.311007 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:51.329930 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0bca8e-c3db-433a-8c82-fc2c16f187d7 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:51.342823 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:51.388279 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:51.389140 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:51.389418 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:51.389687 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] PATH_INFO: `/v3/projects/9ec071fe5a7c47e89b6aef73529f9bc6` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:51.390919 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec071fe5a7c47e89b6aef73529f9bc6)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:51.393373 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:51.518215 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec071fe5a7c47e89b6aef73529f9bc6', 'request_id': 'req-544d5999-8da2-40d7-81bd-d6ae30e86852'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:51.524428 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec071fe5a7c47e89b6aef73529f9bc6', 'request_id': 'req-544d5999-8da2-40d7-81bd-d6ae30e86852'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:51.529991 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec071fe5a7c47e89b6aef73529f9bc6', 'request_id': 'req-544d5999-8da2-40d7-81bd-d6ae30e86852'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:51.637148 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] The token cache is being invalidate because project 9ec071fe5a7c47e89b6aef73529f9bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:46:51.637509 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:51.638125 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-544d5999-8da2-40d7-81bd-d6ae30e86852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:46:51.639679 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1440/2886] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:46:51 2022] DELETE /identity/v3/projects/9ec071fe5a7c47e89b6aef73529f9bc6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 21:46:51.645606 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f943a-8436-45d7-b9a9-a766790d35be None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:51.646105 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f943a-8436-45d7-b9a9-a766790d35be None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:51.646231 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f943a-8436-45d7-b9a9-a766790d35be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:51.664800 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-de5f943a-8436-45d7-b9a9-a766790d35be None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:51.666065 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-de5f943a-8436-45d7-b9a9-a766790d35be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:51.729503 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1447/2887] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:51.739294 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80643ef1-93ef-4d81-8b9a-e5c18d410282 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:51.739582 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80643ef1-93ef-4d81-8b9a-e5c18d410282 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:51.739901 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80643ef1-93ef-4d81-8b9a-e5c18d410282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:51.759017 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-80643ef1-93ef-4d81-8b9a-e5c18d410282 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:51.760285 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-80643ef1-93ef-4d81-8b9a-e5c18d410282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:51.768170 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1441/2888] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:51.772543 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f943a-8436-45d7-b9a9-a766790d35be None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:51.782579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f943a-8436-45d7-b9a9-a766790d35be None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:51.788148 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:51.788990 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:51.789271 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:51.789544 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:51.790283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:46:51.790589 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:51.792242 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:51.806495 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1448/2889] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 21:46:51.812463 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80643ef1-93ef-4d81-8b9a-e5c18d410282 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:51.823086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80643ef1-93ef-4d81-8b9a-e5c18d410282 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:51.827922 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:51.828598 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:51.828888 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:51.829165 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:51.829917 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:51.831876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:51.925206 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1442/2890] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:46:51 2022] POST /identity/v3/projects => generated 369 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:51.930398 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:51.940737 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6b98c-36bb-429c-a87d-df93e03b4f8d admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:51.997425 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:51.998082 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:51.998363 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:51.998634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:51.999308 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:52.001223 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:52.090092 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1449/2891] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:46:51 2022] POST /identity/v3/users => generated 285 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:46:52.094876 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:52.106593 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca3ac4-58b5-4518-bfec-481e662f76fc admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:52.111268 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:52.111961 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:52.112243 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:52.112517 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:52.113236 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:52.114810 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:52.125254 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1443/2892] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:52.129292 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:52.139361 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d048a8a-19bc-4ace-82ae-5c83e99de216 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:52.143915 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a36b04-91bd-4fc7-b5a8-22a264174182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:52.144562 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a36b04-91bd-4fc7-b5a8-22a264174182 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:52.145340 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a36b04-91bd-4fc7-b5a8-22a264174182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:52.145613 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a36b04-91bd-4fc7-b5a8-22a264174182 admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:52.146322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a36b04-91bd-4fc7-b5a8-22a264174182 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:52.147930 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a36b04-91bd-4fc7-b5a8-22a264174182 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:52.157625 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1450/2893] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:46:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:46:52.162301 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:52.172051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b817a37c-16a0-4b99-b1d0-7812a374ebf9 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:52.176665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:52.177570 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:52.177848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:52.178124 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] PATH_INFO: `/v3/projects/633928e23df14e53998ef1ce62f6587a/users/4e4993885b8f4216bbc5619895e613fc/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:52.192226 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] RBAC: Authorizing `identity:create_grant(project_id=633928e23df14e53998ef1ce62f6587a, user_id=4e4993885b8f4216bbc5619895e613fc, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:52.195127 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:52.256216 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1444/2894] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:46:52 2022] PUT /identity/v3/projects/633928e23df14e53998ef1ce62f6587a/users/4e4993885b8f4216bbc5619895e613fc/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:46:52.266644 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed590925-9b25-47d3-b07d-ac798705e2fe None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:52.266936 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed590925-9b25-47d3-b07d-ac798705e2fe None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:52.267219 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed590925-9b25-47d3-b07d-ac798705e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:52.295715 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-ed590925-9b25-47d3-b07d-ac798705e2fe None None] MFA Rules not processed for user `4e4993885b8f4216bbc5619895e613fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:46:52.296956 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-ed590925-9b25-47d3-b07d-ac798705e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:52.338617 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:52.361616 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1451/2895] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:46:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:46:52.373105 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:46:52.375128 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:46:52.422715 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:46:52.434665 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6b6ad4-1516-47ca-a69d-87b2978e2725 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:46:52.438487 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a9f43-5cd7-4c16-a89b-3cff1d484b11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:46:52.439213 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a9f43-5cd7-4c16-a89b-3cff1d484b11 service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:46:52.439524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a9f43-5cd7-4c16-a89b-3cff1d484b11 service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:46:52.439920 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a9f43-5cd7-4c16-a89b-3cff1d484b11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:46:52.449740 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a9f43-5cd7-4c16-a89b-3cff1d484b11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:46:52.451231 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a9f43-5cd7-4c16-a89b-3cff1d484b11 service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:46:52.461458 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1445/2896] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:46:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:47:02.405411 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=8 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:02.662471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=7 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:28.858117 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed590925-9b25-47d3-b07d-ac798705e2fe None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:28.918778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed590925-9b25-47d3-b07d-ac798705e2fe None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:28.923427 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:28.924317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:28.924604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:28.924873 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] PATH_INFO: `/v3/users/4e4993885b8f4216bbc5619895e613fc` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:28.926050 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e4993885b8f4216bbc5619895e613fc)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:28.928121 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:29.087638 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e4993885b8f4216bbc5619895e613fc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:29.101210 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e4993885b8f4216bbc5619895e613fc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:29.152919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e4993885b8f4216bbc5619895e613fc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:29.153531 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e4993885b8f4216bbc5619895e613fc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:29.154134 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e4993885b8f4216bbc5619895e613fc'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:29.172312 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1452/2897] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:47:28 2022] DELETE /identity/v3/users/4e4993885b8f4216bbc5619895e613fc => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:47:29.177871 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:29.186882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:29.186882 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:29.197267 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:29.198481 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:29.260859 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1446/2898] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:29.271932 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:29.273369 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:29.338535 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bb8d9-3ab0-42fe-ad31-e33a82e57b29 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:29.342165 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1cbaf-16d3-4a7f-ad20-80f333278b20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:29.346053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1cbaf-16d3-4a7f-ad20-80f333278b20 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:29.346053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1cbaf-16d3-4a7f-ad20-80f333278b20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:29.346053 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1cbaf-16d3-4a7f-ad20-80f333278b20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:29.352703 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1cbaf-16d3-4a7f-ad20-80f333278b20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:29.354137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1cbaf-16d3-4a7f-ad20-80f333278b20 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:29.363466 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1453/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:47:29.818319 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:29.820157 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:29.834019 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa9965-e985-4fab-91c9-b706baeeb817 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:29.839297 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:29.840251 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:29.840475 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:29.840750 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] PATH_INFO: `/v3/projects/633928e23df14e53998ef1ce62f6587a` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:29.842808 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=633928e23df14e53998ef1ce62f6587a)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:29.844769 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:29.948395 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633928e23df14e53998ef1ce62f6587a', 'request_id': 'req-d385cac4-f498-4ebe-b878-2299df382263'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:29.954609 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633928e23df14e53998ef1ce62f6587a', 'request_id': 'req-d385cac4-f498-4ebe-b878-2299df382263'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:29.961153 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633928e23df14e53998ef1ce62f6587a', 'request_id': 'req-d385cac4-f498-4ebe-b878-2299df382263'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:30.026569 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] The token cache is being invalidate because project 633928e23df14e53998ef1ce62f6587a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80757) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:47:30.026851 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:30.027460 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-d385cac4-f498-4ebe-b878-2299df382263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:30.029028 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1447/2900] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:47:29 2022] DELETE /identity/v3/projects/633928e23df14e53998ef1ce62f6587a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:47:30.033919 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b329203-1e85-4958-b958-da4c4dc2c3e5 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.034402 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b329203-1e85-4958-b958-da4c4dc2c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.045929 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b329203-1e85-4958-b958-da4c4dc2c3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.052405 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-5b329203-1e85-4958-b958-da4c4dc2c3e5 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:30.053604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-5b329203-1e85-4958-b958-da4c4dc2c3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:30.071487 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:30.113241 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1454/2901] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:30.172055 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb9be-17c7-4cfe-aa86-09b2bc1fd7a4 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.172338 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb9be-17c7-4cfe-aa86-09b2bc1fd7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.172600 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fb9be-17c7-4cfe-aa86-09b2bc1fd7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.191098 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-d38fb9be-17c7-4cfe-aa86-09b2bc1fd7a4 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:30.192946 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-d38fb9be-17c7-4cfe-aa86-09b2bc1fd7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:30.200690 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1448/2902] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:30.204889 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b329203-1e85-4958-b958-da4c4dc2c3e5 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:30.214634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b329203-1e85-4958-b958-da4c4dc2c3e5 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:30.219268 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:30.220006 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.220270 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.220532 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.221260 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80756) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 21:47:30.221569 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:30.223192 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:30.234711 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1455/2903] 10.222.0.42 () {58 vars in 1152 bytes} [Thu May 26 21:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 21:47:30.239527 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fb9be-17c7-4cfe-aa86-09b2bc1fd7a4 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:30.248643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fb9be-17c7-4cfe-aa86-09b2bc1fd7a4 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:30.253232 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:30.253878 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.254168 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.254468 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.255736 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:30.257643 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:30.360578 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1449/2904] 10.222.0.42 () {60 vars in 1152 bytes} [Thu May 26 21:47:30 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 21:47:30.365505 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:30.375809 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e73ae-c888-4b1f-94b9-8676cb8b19c3 admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:30.435437 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:30.436147 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.436432 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.436706 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] PATH_INFO: `/v3/users` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.437416 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:30.439351 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:30.536317 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1456/2905] 10.222.0.42 () {60 vars in 1143 bytes} [Thu May 26 21:47:30 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 21:47:30.541290 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:30.553147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eac82d5-6159-4203-825f-5cb1d915cf92 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:30.557953 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:30.558672 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.558936 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.559212 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.560020 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:30.561617 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:30.573590 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1450/2906] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 21:47:30.577782 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:30.588651 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fc814-d9a9-4598-8a68-c71dfcb8367a admin admin] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:30.593538 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9baa69-64b6-4c2e-a30a-b8c055d9508f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:30.594217 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9baa69-64b6-4c2e-a30a-b8c055d9508f admin admin] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.594506 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9baa69-64b6-4c2e-a30a-b8c055d9508f admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.594820 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9baa69-64b6-4c2e-a30a-b8c055d9508f admin admin] PATH_INFO: `/v3/roles` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.595585 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9baa69-64b6-4c2e-a30a-b8c055d9508f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:30.597244 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9baa69-64b6-4c2e-a30a-b8c055d9508f admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:30.608449 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1457/2907] 10.222.0.42 () {58 vars in 1121 bytes} [Thu May 26 21:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 21:47:30.612722 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:30.623432 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e49263-f281-4118-bbbc-4d362d4eaa2c admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:30.628065 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:30.628947 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.629233 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.630075 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] PATH_INFO: `/v3/projects/df9771acff0a4257add6cd30be632e7d/users/451a2a9257854273845a45f544cd2b39/roles/1183a502796d46e580a92e3c20f04103` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.644456 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=df9771acff0a4257add6cd30be632e7d, user_id=451a2a9257854273845a45f544cd2b39, role_id=1183a502796d46e580a92e3c20f04103)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:30.647768 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:30.724380 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1451/2908] 10.222.0.42 () {60 vars in 1482 bytes} [Thu May 26 21:47:30 2022] PUT /identity/v3/projects/df9771acff0a4257add6cd30be632e7d/users/451a2a9257854273845a45f544cd2b39/roles/1183a502796d46e580a92e3c20f04103 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:47:30.734276 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640d1d1-a233-48c7-88f1-d9bc39287c1f None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.734600 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640d1d1-a233-48c7-88f1-d9bc39287c1f None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.734879 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f640d1d1-a233-48c7-88f1-d9bc39287c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.762450 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-f640d1d1-a233-48c7-88f1-d9bc39287c1f None None] MFA Rules not processed for user `451a2a9257854273845a45f544cd2b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:30.763710 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f640d1d1-a233-48c7-88f1-d9bc39287c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:30.828151 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1458/2909] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:47:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:30.839773 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:30.841501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:30.864372 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:30.901699 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e73586-d3c1-475d-acc0-8474947461aa admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:30.905486 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6cf996e1fdb4a458d06bfd23d082dec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:30.907406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:30.907406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:30.907406 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:30.916739 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:30.918186 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:30.928513 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1452/2910] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:47:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:47:32.765328 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:32.765917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:32.765917 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:32.784305 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] MFA Rules not processed for user `451a2a9257854273845a45f544cd2b39`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:32.785545 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:32.792831 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1459/2911] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:47:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:32.833536 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:32.836680 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:32.895018 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a353c4-1444-49ca-bb74-98e0847ddc82 service cinder] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:32.899326 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11bdc1-040a-4340-b7ce-f0b3e2d3c553 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a98415ce308d41f6b8d5db6ca3f2ff19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:32.900021 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11bdc1-040a-4340-b7ce-f0b3e2d3c553 service glance] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:32.900304 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11bdc1-040a-4340-b7ce-f0b3e2d3c553 service glance] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:32.900571 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11bdc1-040a-4340-b7ce-f0b3e2d3c553 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:32.902476 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7d11bdc1-040a-4340-b7ce-f0b3e2d3c553 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:32.988183 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11bdc1-040a-4340-b7ce-f0b3e2d3c553 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:32.991282 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d11bdc1-040a-4340-b7ce-f0b3e2d3c553 service glance] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:33.013896 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1453/2912] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:47:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:47:33.588621 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:33.590226 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:33.604705 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:33.658882 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91955a8c-f1e1-41e9-b7f5-163d7a80bc71 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:33.663850 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:33.664714 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:33.665115 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:33.665469 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:33.667839 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:33.685646 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:33.687113 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:33.703721 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1460/2913] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:47:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:47:43.002220 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=71 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:43.694245 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=21 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:51.882130 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f83483-8b74-42ff-94c5-a6b4ca8949fe None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:51.882594 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f83483-8b74-42ff-94c5-a6b4ca8949fe None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:51.882708 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f83483-8b74-42ff-94c5-a6b4ca8949fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:51.900702 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-04f83483-8b74-42ff-94c5-a6b4ca8949fe None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:51.901907 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-04f83483-8b74-42ff-94c5-a6b4ca8949fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:51.964744 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1454/2914] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:47:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:52.137080 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:52.138528 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:52.207172 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e555df-2b51-466d-b52e-cd9ec45b4265 service neutron] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:52.212658 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:52.213297 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:52.213579 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:52.213851 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:52.221778 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:52.223350 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:52.232183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1461/2915] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:47:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:47:53.047506 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315eabf-99e0-44f0-b21a-0a161c0c7859 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:53.048166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315eabf-99e0-44f0-b21a-0a161c0c7859 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:53.048166 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315eabf-99e0-44f0-b21a-0a161c0c7859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:53.067721 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-7315eabf-99e0-44f0-b21a-0a161c0c7859 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:53.069501 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-7315eabf-99e0-44f0-b21a-0a161c0c7859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:53.076987 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1455/2916] 10.222.0.42 () {58 vars in 967 bytes} [Thu May 26 21:47:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:53.091184 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:53.102713 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb10ca1-c96f-4c31-815d-5d7a9f1e0fc8 service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:53.108357 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:53.109027 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:53.109311 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:53.109591 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:53.119300 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:53.120947 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:53.129634 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1462/2917] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:47:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:47:53.712558 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f7bf6-2310-4535-a51e-67ba2a936174 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:53.712853 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f7bf6-2310-4535-a51e-67ba2a936174 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:53.713141 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f7bf6-2310-4535-a51e-67ba2a936174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:53.730685 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-db4f7bf6-2310-4535-a51e-67ba2a936174 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:53.731848 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-db4f7bf6-2310-4535-a51e-67ba2a936174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:53.739244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1456/2918] 10.222.0.42 () {58 vars in 1009 bytes} [Thu May 26 21:47:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:53.751661 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:53.763100 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bec044-05e5-4e4a-a7f5-459a7a8385dd service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:53.768129 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:53.768769 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:53.769057 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:53.769349 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:53.777494 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:53.779081 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:53.788283 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1463/2919] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:47:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:47:54.518030 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1ea2e-8bed-4a98-ba91-ce938be89733 None None] REQUEST_METHOD: `POST` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:54.518524 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1ea2e-8bed-4a98-ba91-ce938be89733 None None] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:54.518626 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1ea2e-8bed-4a98-ba91-ce938be89733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:54.536693 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.auth.core [None req-4ca1ea2e-8bed-4a98-ba91-ce938be89733 None None] MFA Rules not processed for user `67316e32d66342d58684c1ee3e5894c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80757) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:54.538465 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-4ca1ea2e-8bed-4a98-ba91-ce938be89733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:54.545676 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1457/2920] 10.222.0.42 () {58 vars in 967 bytes} [Thu May 26 21:47:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:54.559004 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:54.570110 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5595a-9bde-47f1-94f0-ff8d48279715 service swift] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:54.575371 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51debe0-e060-4db1-b74c-31b1003df1d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ededf6d88dee4745978c8566940b2180', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:54.576043 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51debe0-e060-4db1-b74c-31b1003df1d1 service swift] REQUEST_METHOD: `GET` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:54.576327 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51debe0-e060-4db1-b74c-31b1003df1d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:54.576604 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51debe0-e060-4db1-b74c-31b1003df1d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:54.586175 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51debe0-e060-4db1-b74c-31b1003df1d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:54.587951 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51debe0-e060-4db1-b74c-31b1003df1d1 service swift] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:54.597282 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1464/2921] 10.222.0.42 () {60 vars in 1384 bytes} [Thu May 26 21:47:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 21:47:57.214667 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1ea2e-8bed-4a98-ba91-ce938be89733 None None] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:57.281720 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1ea2e-8bed-4a98-ba91-ce938be89733 None None] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:57.286515 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:57.288057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:57.288335 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:57.288607 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] PATH_INFO: `/v3/users/451a2a9257854273845a45f544cd2b39` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:57.289799 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451a2a9257854273845a45f544cd2b39)` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:57.292057 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:57.445471 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451a2a9257854273845a45f544cd2b39'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:57.457376 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451a2a9257854273845a45f544cd2b39'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:57.501147 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451a2a9257854273845a45f544cd2b39'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:57.501884 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451a2a9257854273845a45f544cd2b39'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:57.502551 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.notifications [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451a2a9257854273845a45f544cd2b39'} {{(pid=80757) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:57.517202 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1458/2922] 10.222.0.42 () {58 vars in 1223 bytes} [Thu May 26 21:47:57 2022] DELETE /identity/v3/users/451a2a9257854273845a45f544cd2b39 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:47:57.522541 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] REQUEST_METHOD: `POST` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:57.523023 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:57.523126 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:57.540793 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.auth.core [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] MFA Rules not processed for user `ef098ca75a4b4231b612fc38df12339d`. Rule list: `[]` (Enabled: `True`). {{(pid=80756) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 21:47:57.541997 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:57.603063 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1465/2923] 10.222.0.42 () {58 vars in 957 bytes} [Thu May 26 21:47:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 21:47:57.614528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:57.614528 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:57.620955 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:57.674330 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73437dfb-1787-46de-9737-4305ec4598a0 admin admin] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:57.678045 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e757ff94b7474fd99a1c7221be19c64d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:57.678691 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:57.678975 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:57.679253 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:57.689228 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:57.690659 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:57.702269 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1459/2924] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:47:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:47:57.946183 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] Authenticating user token {{(pid=80756) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:47:57.947688 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.fernet_utils [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80756) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:47:57.961804 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426efa8c-6c0f-4f04-a70b-b2f3ebaa4121 None None] Validating token access rules against request {{(pid=80756) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:47:57.966563 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef098ca75a4b4231b612fc38df12339d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '213a059b002f40c1bddeb6d9f1da1459', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80756) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:47:57.967880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:47:57.967880 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:47:57.968137 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] PATH_INFO: `/v3/projects/df9771acff0a4257add6cd30be632e7d` {{(pid=80756) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:47:57.969262 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df9771acff0a4257add6cd30be632e7d)` {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:47:57.971208 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] RBAC: Authorization granted {{(pid=80756) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:47:58.075322 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:47:58.078215 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df9771acff0a4257add6cd30be632e7d', 'request_id': 'req-12f01cc5-faa4-47af-894c-56c90c2d74a0'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:58.085159 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df9771acff0a4257add6cd30be632e7d', 'request_id': 'req-12f01cc5-faa4-47af-894c-56c90c2d74a0'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:58.094772 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df9771acff0a4257add6cd30be632e7d', 'request_id': 'req-12f01cc5-faa4-47af-894c-56c90c2d74a0'} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:58.146988 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] The token cache is being invalidate because project df9771acff0a4257add6cd30be632e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80756) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 21:47:58.147307 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:58.147954 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG keystone.notifications [None req-12f01cc5-faa4-47af-894c-56c90c2d74a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80756) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 21:47:58.149534 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: [pid: 80756|app: 0|req: 1466/2925] 10.222.0.42 () {58 vars in 1232 bytes} [Thu May 26 21:47:57 2022] DELETE /identity/v3/projects/df9771acff0a4257add6cd30be632e7d => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 21:48:07.721579 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=24 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:48:08.151596 dv-cinder-828805-3-iscsi devstack@keystone.service[80756]: DEBUG dbcounter [-] [80756] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80756) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 21:50:13.862466 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] Authenticating user token {{(pid=80757) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 21:50:13.863890 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:50:14.002661 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39576e-930d-47e3-8308-11a9927337ce service neutron] Validating token access rules against request {{(pid=80757) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 21:50:14.061051 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cd4dc-0062-4ef3-98f1-555326b46365 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd73860db589439da5178e7010d567d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c2f571f2f8f4a82978ad8a99e7a9bf3', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80757) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 21:50:14.061706 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cd4dc-0062-4ef3-98f1-555326b46365 service placement] REQUEST_METHOD: `GET` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 21:50:14.061993 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cd4dc-0062-4ef3-98f1-555326b46365 service placement] SCRIPT_NAME: `/identity` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 21:50:14.062270 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cd4dc-0062-4ef3-98f1-555326b46365 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80757) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 21:50:14.064086 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.fernet_utils [None req-2a9cd4dc-0062-4ef3-98f1-555326b46365 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80757) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 21:50:14.075244 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cd4dc-0062-4ef3-98f1-555326b46365 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 21:50:14.076894 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cd4dc-0062-4ef3-98f1-555326b46365 service placement] RBAC: Authorization granted {{(pid=80757) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 21:50:14.091040 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: [pid: 80757|app: 0|req: 1460/2926] 10.222.0.42 () {60 vars in 1365 bytes} [Thu May 26 21:50:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 21:50:24.082564 dv-cinder-828805-3-iscsi devstack@keystone.service[80757]: DEBUG dbcounter [-] [80757] Writing DB stats keystone:SELECT=89 {{(pid=80757) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}